MLK’s son recounts racism he faced with his father and why he never felt afraid

Martin Luther King Jr. was very relentless in his fight for equality and justice despite the harassment and threats that came his way while on the line of duty. His family also was also not left out.

But in a recent interview with PEOPLE, the iconic civil rights activist’s son, Martin Luther King III, said he treasured each and every moment he got to spend with his father, even when his trips took them “into the lion’s mouth.”

“I was very fortunate to be able to travel with Dad maybe seven or eight times, and the most important time, however, was that personal time that we spent with him,” human rights advocate Martin Luther King III recalled.

And despite his father’s popularity and influence, King said he always made time for his children. “My brother and I used to go to the YMCA every week when he would go to get exercise, and he taught us how to swim,” the 64-year-old said, adding that they also “used to ride bicycles together in Atlanta.”

But King also recalled the hostilities and racism they experienced when they joined their father on his trips. The 64-year-old made mention of a time they joined the civil rights activist on a trip to St. Augustine, Florida, in 1964. He said his father and some leaders in the American Civil Rights Movement had gone to the city as part of their desegregation campaign efforts. But King recalled the manager of a hotel poured “acid and lye into the pool to keep Black folks from swimming.”

In hindsight, King said he “would have loved” to ask his parents how they determined “it was going to be safe for us as children to accompany Dad.”

“At any moment something could have exploded and something could have happened to us,” he added.

King credited his parents’ courageous efforts to their faith, adding that his faith in his father always made him feel safe. “I always felt safe in the presence of Dad, even when a German shepherd dog approached us one day,” he told PEOPLE in reference to the St. Augustine trip.

“It was three or four of us standing, and the policeman came with the dog. Those dogs had been used in Birmingham [during the integration protests in 1963],” he recalled. “But I never felt afraid because I could grab Dad’s pants leg, and something went through me — a comfort — because I was with my father.”

“Those are just some of the experiences that will always be so meaningful to me,” he said.

Elsewhere in the interview, King also said he fully came to understand the magnitude of his father’s influence and work following his 1968 assassination. He recalled the likes of Aretha Franklin, Harry Belafonte, Sidney Poitier, Sammy Davis Jr. and “every candidate who was running for president” attended his funeral.

“Many were inspired by Dad. Many were involved in helping to raise money, to get the work done,” he said. “But to see all of these individuals and over 200,000 people you know who actually attended his homegoing services, that’s when I realized, ‘Oh my gosh. Dad was incredible.'”

Despite his father’s unfortunate assassination, the surviving members of Martin Luther King Jr.’s family have been relentless in keeping his legacy alive. “I hope that he would be proud of the fact that our family is still engaged,” King told PEOPLE.

“I’m sure he’d be proud of his granddaughter [Yolanda, 13]. She says, ‘I want to continue in my grandfathers’ and grandmothers’ footsteps, but I want to create my own footsteps.’ It’s in her DNA.”