Dolly Parton may not have any children of her own, but she’s always had a big heart for kids, and that maternal instinct dates back to when she was a child herself.

The country legend was one of 12 kids growing up, and in a new appearance on Bunnie Xo‘s Dumb Blonde podcast, she explains that being in a big family meant that the older siblings had to help take care of the younger ones.

“There was only 18 months, two years’ difference in all of our ages,” Parton remembers. “Mama and Daddy got married when Mama was 15, she had her first baby at 16, and they were just born right after another.”

“So any older child had to help with whoever was coming along.”

Read More: Why Dolly Parton Never Had Children

In fact, Parton says her mother would sometimes “assign” infants to specific older siblings, and when she was nine years old, her mom was pregnant with the baby that was supposed to be her special charge.

“The one that was gonna be my baby was little Larry,” she remembers.

“He didn’t live that long, actually. He died at birth,” the singer continues. “But I followed Mama around the whole time she was pregnant. She said that one was gonna be mine. I’d sing to it, kiss it on the belly, but he didn’t make it.”

At just nine years old, Parton hadn’t experienced much death firsthand before, and losing her little brother was a devastating experience.

“It just crushed me,” she recalls.

“I didn’t understand about death and all of that. So that was a real hit for me at that age, that I had lost my baby,” she continues. “I had a guilt thing about it, somehow.”

The tragedy impacted the whole family, but Parton remembers Larry’s death hitting her a little harder than it did her other siblings.

“Because he was supposed to be mine,” she says, “I was … picturing how I was gonna rock him and how I was gonna sing him my songs, you know, and all that.”

As a successful country star, Parton has leveraged her platform into helping children in many different ways. Perhaps most notably, she established her Imagination Library, which provides free books for children up to the age of five in many states across the U.S. and different countries across the world.