His portrayal of sloppy sportswriter Oscar Madison on TV's "The Odd Couple" left viewers laughing but it also gave Klugman the leverage to create a more serious character, the gruff medical examiner in "Quincy M.E." His everyman ethos and comic timing endeared him to audiences and led to a prolific, six-decade acting career that spanned stage, screen and television.
Klugman died Monday at age 90 in suburban Northridge with his wife at his side. His sons called on his fans to embrace their father's tenacious and positive spirit.
"He had a great life and he enjoyed every moment of it, and he would encourage others to do the same," son Adam Klugman said.
The cause of Klugman's death was not immediately known. Adam Klugman said his father had been slowing down in recent years, but wasn't battling cancer, which robbed him of his voice in the 1980s. Klugman taught himself to speak again, and kept working.
He remained popular for decades simply by playing the type of man you could imagine running into at a bar or riding on a subway with - gruff, but down-to-earth, his tie stained and a little loose, a racing form under his arm, a cigar in hand during the days when smoking was permitted.
Off-screen, Klugman owned racehorses and enjoyed gambling, although acting remained his passion.
Despite his on-screen wars with Tony Randall's neat-freak character Felix Unger on "Odd Couple," the show created a friendship between the men that endured after the series ended.
When Randall died in 2004 at age 84, Klugman told CNN: "A world without Tony Randall is a world that I cannot recognize."
The "Odd Couple," which ran from 1970 to 1975, was based on Neil Simon's play about mismatched roommates - divorced New Yorkers who end up living together. The comedy came from their opposite personalities - Klugman playing a writer whose sloppiness consistently irritated the Randall's fussy photographer character. The pairing was so good, the show didn't need constant help from the writers.
"There's nobody better to improvise with than Tony," Klugman said. "A script might say, 'Oscar teaches Felix football.' There would be four blank pages. He would provoke me into reacting to what he did. Mine was the easy part."
Fans and fellow actors agreed it worked, posting clips of their favorite Klugman roles on Twitter and other social networking sites late Monday.
"RIP Jack Klugman. You made my whole family laugh together," actor-director Jon Favreau wrote on Twitter.
"He was a wonderful man and supremely talented actor," wrote actor Max Greenfield, who worked with Klugman several years ago. "He will be missed."
In "Quincy, M.E.," which ran from 1976 to 1983, Klugman played an idealistic, tough-minded medical examiner who tussled with his boss by uncovering evidence of murder in cases where others saw natural causes.
"We had some wonderful writers," he said in a 1987 Associated Press interview. "Quincy was a muckraker, like Upton Sinclair, who wrote about injustices. He was my ideal as a youngster, my author, my hero.
"Everybody said, 'Quincy'll never be a hit.' I said, 'You guys are wrong. He's two heroes in one, a cop and a doctor.' A coroner has power. He can tell the police commissioner to investigate a murder. I saw the opportunity to do what I'd gotten into the theater to do - give a message.
"They were going to do cops and robbers with 'Quincy.' I said, 'You promised me I could do causes.' They said, 'Nobody wants to see that.' I said, 'Look at the success of '60 Minutes.' They want to see it if you present it as entertainment."
For his 1987 role as 81-year-old Nat in the Broadway production of "I'm Not Rappaport," Klugman wore leg weights to learn to shuffle like an elderly man. He said he would wear them for an hour before each performance "to remember to keep that shuffle."
"The guy is so vital emotionally, but physically he can't be," Klugman said. "We treat old people so badly. There is nothing easy about 80."
The son of Russian Jewish immigrants, he was born in Philadelphia and began acting in college at Carnegie Institute of Technology. After serving in the Army during World War II, he went on to summer stock and off-Broadway, rooming with fellow actor Charles Bronson as both looked for paying jobs. He made his Broadway debut in 1952 in a revival of "Golden Boy."
His film credits included Sidney Lumet's "12 Angry Men" and Blake Edwards' "Days of Wine and Roses" and an early television highlight was appearing with Humphrey Bogart and Henry Fonda in a production of "The Petrified Forest." His performance in the classic 1959 musical "Gypsy" brought him a Tony nomination for best featured (supporting) actor in a musical.
He also appeared in several episodes of "The Twilight Zone," including a memorable 1963 one in which he played a negligent father whose son is seriously wounded in Vietnam. His other TV shows included "The Defenders" and the soap opera "The Greatest Gift."
Throat cancer took away his raspy voice for several years in the 1980s. When he was back on the stage for a 1993 revival of "Three Men on a Horse," the AP's review said, "His voice may be a little scratchy but his timing is as impeccable as ever."
"The only really stupid thing I ever did in my life was to start smoking," he said in 1996. Seeing people smoking in television and films "disgusts me, it makes me so angry - kids are watching," he said.
In his later years, he guest-starred on TV series including "Third Watch" and "Crossing Jordan" and appeared in a 2010 theatrical film, "Camera Obscura."
Klugman's hobby was horse racing and he eventually took up raising them, too.
"I always loved to gamble," he said. "I never got close to a horse. Fate dealt me a terrible blow when it gave me a good horse the first time out. I thought how easy this is.
"Now I love being around them."
A horse Klugman co-owned, Jacklin Klugman, finished third in 1980's Kentucy Derby and fourth in that year's Preakness Stakes.
Klugman's wife, actress-comedian Brett Somers, played his ex-wife, Blanche, in the "Odd Couple" series. The couple, who married in 1953 and had two sons, Adam and David, had been estranged for years by the time of her death in 2007.
In February 2008, at age 85, Klugman married longtime girlfriend Peggy Crosby, who was by his side when he died Monday.
His attorney Larry Larson wrote in an email that Klugman is also survived by two grandchildren and that memorial services have not been set.
Biographical material in this story was written by former AP staffer Polly Anderson, and AP sports writer Beth Harris contributed to this report.
Copyright 2012 The Associated Press.