A 65th Street townhouse in Manhattan is on the market for $32 million. for The Wall Street Journal

For 70 years, the China Institute in America has served up Far East culture from its quaint, wood-paneled mansion on the Upper East Side. Now, it has put its home on the market for $32 million and is moving downtown.

The institute's chairman, Chien Chung Pei, an architect and son of I.M. Pei, said it would soon move into larger renovated space it bought for $18.3 million.

The space is a commercial condominium in the Financial District with room for large galleries, multimedia displays and ample classrooms for lessons in Mandarin Chinese.

It is four times the size of the institute's current home, on East 65th Street near Park Avenue. The move will make the 35-foot wide Federal-style brick and stone mansion, completed in 1905, one of a very few extra-wide townhouses on the market in the city, "all in need of renovation and a little bit of TLC," according to Corcoran Group listing broker Carrie Chiang.

The mansion, designed by Charles A. Platt for a physiologist, Frederic S. Lee, has considerable original detail, from a curved main staircase with a decorative iron railing to a library with an exposed wood beam ceiling and paneled walls.

The plan to expand the institute has been in the works for 10 years, said Acting President Thomas B. Moore, since the institute, once a cultural center for anticommunist Chinese leaders, began sponsoring an annual "executive summit" bringing together Chinese and American business and political figures.

China provides some funding for the Chinese language classes at the institute, through a program called the Confucius Institute, he said.

The institute was founded in 1926 with reparations paid by the Chinese government to the U.S. for the Boxer Rebellion, a violent anti-foreign and anti-Christian movement in China around 1900.

The mansion was purchased in 1944 with funds provided by the foundation of Henry Luce, the co-founder and editor in chief of Time Inc. and the son of a missionary in China. He served as president and chairman of the board of the institute.

In the 1950s, the institute became a center of anticommunist opposition and support for the Chinese Nationalist Government. In 1951, Dean Rusk, then an assistant secretary of state and later the secretary of state railed against the Communist government at an institute dinner as a "colonial Russian government."

That was a few years after hundreds of Columbia University students protested against a lecture series on China co-sponsored by the institute. The students complained at the time that the series was one-sided in favor of the nationalist government.

By the 1960s, the institute created two tiny permanent art exhibition spaces on the mansion's ground floor, teaching Chinese cooking to Americans raised on egg rolls and chop suey. The kitchen has since been turned into a classroom.

When relations between the U.S. and China opened up at the end of the 1970s, the institute rekindled its cultural connections with China, Mr. Pei said.

In 2012, after a long search, the institute purchased 41,000 square feet of space at 40 Rector St., a 19-story office building near the World Trade Center, including a ground-floor gallery and the entire second floor.

The second-floor space is occupied by the Civilian Complaint Review Board, an independent city agency set up to handle complaints about the New York Police Department. But the agency is due to move out in the spring, Mr. Moore said.

The institute might relocate the traditional Chinese rock garden in the mansion's rear yard to the new office building.

The mansion has about 9,800 square feet over four floors, plus the 1,050-square-foot rock garden.

Write to Josh Barbanel at josh.barbanel@wsj.com