I should be working on my memoir today. There’s no reason I can’t pick up where I left off yesterday, with my teenage self, in the woods, at night. The toes of a hiking boot—and then the toes of the other boot—are lit in the beam of a flashlight, as I walk from the staff cabin at the wilderness camp to the bathrooms. That walk happened, but the boot in the beam of the flashlight—as probable as it is—is not, strictly speaking, remembered. I made that part up. Chances are I stumbled through the woods thinking thoughts that had nothing to do with whatever my feet were up to.

Then why did I write it that way, and why do I think it’s important to keep it? The short answer to both questions is: it serves a literary purpose. As I sat with the scene yesterday, the story needed slowing down, to be grounded. I can’t fully explain what happens when I’m writing and I follow an impulse that feels right. I don’t always understand why a story makes its demands, but when it does, I’ve learned to go with it.

In the scheme of things, it’s a minor fiction. I’m not making flagrant claims, like Binjamin Wilkomirski, author of Fragments did, in his memoir about a Polish, Jewish boy who survives a Nazi death camp. It turned out he isn’t Jewish, or even Polish, let alone a Holocaust survivor. I was, without a doubt, the girl in the woods, taking a break from the staff cabin and its roaring fire, the cigarettes and marijuana, and a jug of wine being passed around. No one is going to argue about whether I noticed that my flashlight lit up the toe of my boot. Yet something still bothers me. I’m feeling nervous about the liberty I’ve taken with “what really happened.”

Is it dishonest to tinker, in however a minor way, with the stuff of my life? I recreate conversations I can’t possibly remember verbatim, change names, and rely on memories I’ve gone over and over again, surely distorting them in the process. On the other hand, it’s no use worrying about how people will receive a book that may never see print. But what I’m more afraid of than being caught out is that the bit of wisdom I’m trying to uncover is actually self-deception.

Vivian Gornick, author of the mother-daughter memoir, Fierce Attachments, and the well-loved memoir-writing book, The Situation and the Story, sees deviations from what actually happened as memoirists’ prerogative. “What actually happened is only raw material; what the writer makes of what happened is all that matters,” she wrote in an essay, defending her use of composite characters. She cites memoirs acknowledged to be masterpieces by authors as diverse as Edmund Gosse, who recounted conversations that supposedly took place when he was eight years old, to George Orwell, who was denounced for inaccuracies in his account of his school days in “Such, Such Were the Joys.”

In Gornick’s view, the problem is not with the memoirists but with readers’ expectations. “Memoirs belong to the category of literature, not of journalism. It is a misunderstanding to read a memoir as though the writer owes the reader the same record of literal accuracy that is owed in newspaper reporting or in literary journalism.” She doesn’t go so far as to condone memoirists like Wilkomirski, who invent pasts they never had. She makes a distinction between inventing a narrative out of whole cloth, as Wilkomirski did, and composing, which is what she did when working with, as she says, “a narrative drawn entirely from the materials of my own experience.”

Other memoirists, too, are unapologetic about shaping narratives out of the raw materials of their lives. David Sedaris was trashed in The New Republic for making stuff up about ten years ago, around the time James Frey, Stephen Glass, and Jayson Blair were being publicly shamed for doing what appeared, to his critics, to be the same thing. Did the fact that he survived where the others fell have to do with being a humor writer, which makes us more forgiving? Or was it because his exaggerations were more along the lines of what Gornick calls “composing” from experience versus inventing a past that wasn’t what really happened?

There’s that phrase again, “what really happened,” the idea that keeps stopping me when I’m feeling most connected to my work. I’m haunted by Margo Jefferson, who in her memoir, Negroland, writes: “I think it’s too easy to recount your unhappy memories when you write about yourself. You bask in your own innocence. You revere your grief. You arrange your angers at their most becoming angles.”

Am I doing that?

How can I not? My temperament, ego, and self-image are all part of the story. And my life is not a story until I find what the story is. Like Hansel and Gretel, I wander in the woods, pausing to shine a light on my childhood and adolescence while taking stock of the world around me and my place in it, then and now.

To avoid confusion, I could call what I’m writing “realish,” as Sedaris does, or “based on a true story,” as they say in the movies. Or I could continue to keep Vivian Gornick nearby and remind myself of what a memoirist owes the reader, which, as she writes, “is the ability to persuade that the narrator is trying, as honestly as possible, to get to the bottom of the experience at hand.”

I don’t know why I’m a memoirist and not a fiction writer—I often wish it were otherwise. But that’s the way it is. If I want whatever bit of truth I have achieved in my work to reach a reader, I have to be faithful to the story I’m telling. Ironically, the demands of literature—for drama, narrative drive, and conflict—are what stop me from being wholly self-serving in the way Jefferson derides. Self-pity makes for a boring read.