Whitmore attended Yale, where he joined the Yale Drama School Players and co-founded the Yale radio station. After serving in World War II with the Marines, he did some work in stock and then debuted on Broadway in 1947's Command Decision. He entered films in 1949, going on to play key supporting roles; occasionally, he also played leads. For his work in Battleground (1949), his second film, he received a Best Supporting Actor Oscar nomination. He starred in the early '60s TV series "The Law and Mr. Jones." He won much acclaim for his work in the one-man stage show Give 'Em Hell, Harry!, in which he played Harry Truman; he reprised the role in the 1975 screen version, for which he received a Best Actor Oscar nomination. After 1980 his screen appearances were infrequent. He is the father of actor James Whitmore Jr.