Joe Jackson, patriarch and taskmaster of the Jackson family, has died. Last week, Jackson was hospitalized with terminal pancreatic cancer. TMZ reports that he died late last night in Los Angeles. Jackson was 89.

Jackson was born in Arkansas, the grandson of a slave. When his parents broke up, he moved with his mother to the Chicago suburbs of Indiana. After dropping out of high school, Jackson became a steelworker and an amateur boxer. He married his second wife Katherine in 1949, and they started having children soon after. Together, they had 10 children.

Jackson was a musician himself, playing guitar in a short-lived blues band called the Falcons. But he made his impact on music history when he put five of his sons into a group that soon became known as the Jackson 5. A harsh taskmaster, Jackson put his sons through constant, rigorous rehearsals, often physically and mentally abusing them. (In later years, Jackson refused to apologize for abusing his kids, claiming that his approach is the reason the Jacksons were successful.)

Under Jackson’s supervision, the Jackson 5 toured the country and released the local hit “Big Boy” before signing with Motown in 1969. He and the family moved to California, and the Jackson 5 became an immediate sensation, getting to #1 with their very first single, the pop masterpiece “I Want You Back.” Soon after, Jackson moved much of the family to Las Vegas, booking the non-Jackson 5 members, including daughter Janet, as performers in casinos.

Jackson’s legacy is dark. He has a place in music history, but it’s for the accomplishments of his children, not for anything that he did himself. It’s entirely possible that we never would’ve heard Michael and Janet Jackson without Joe’s abusive, domineering parenting. But it’s just as likely that he scarred his own kids for life, making them much less happy and healthy even as he ushered them to fame.