When I was a kid I grew up in one, completely average house, there was no history of tragedy, in fact, we were it's second owners and the previous family had no negative experiences.
I remember that growing up that I didn't like being left alone in the sitting room. I didn't object much because my parents were only a metre away but the corner on the far right, right at the bottom, had something off about it.
I would regularly find myself being pulled into a kind of trance where the room would grow dark and then the corner seemed to be blinding me with strobe lighting. I could see them when I was like that, the little people in there. They had long sharp fingers, sharper than anything I could imagine, still visible but so thin you'd question where they ended, and the boxes. Millions of tiny white boxes. Perfect cubes, spilling out of the corner around them.
They had a certain smell too, metallic and sharp, everything about them was sharp. The air would taste like metal too. Maybe that's what the boxes were.
They didn't always notice me, but they didn't like children, I knew that. Sometimes I would feel them scratching my arms, but there was never any trace of them after I snapped out of it, with the exception of the sharp smell, fading away but still. My parents couldn't see or smell anything. To them it was just a corner.
Sometimes I wonder if it was even real, but it happened too regularly to ignore. By the time I was 6 I couldn't see or feel them anymore. I think I'm too old to see them now.
I've had no physical damage but I'm well aware of how uncomfortable it can be to experience something such as this and be uncertain as to how to proceed. While I can't say I have the answer to this question, I would be happy to discuss the encounters and I would greatly appreciate knowing whether or not these experiences were common to anyone else's lives.
There's a series of books cowritten by Terry Pratchett and Stephen Baxter called the "Long Earth" about the effect on society in the face of a discovery that enables anyone to travel between practically infinite dimensions. Along the way, it is discovered that there have been beings with this capability all along explaining sitings of many entities considered cryptids or simply mythological until then. They actually come upon at least one entity that seems very much like what you have described. Granted, this is fiction, but Pratchett always drew very heavily from folk lore. It may be an interesting read for you:-)