A group of Philadelphians had joined together in 1740 to erect a great preaching hall for the traveling evangelist George Whitefield, who tours the American colonies delivering open air sermons.

The building is designed and built by Edmund Woolley and is the largest building in the city at the time.

It is initially planned to serve as a charity school as well; however, a lack of funds forces plans for the chapel and school to be suspended.

Benjamin Franklin, eager to create a school to educate future generations, in the fall of 1749 had circulated a pamphlet titled "Proposals for the Education of Youth in Pennsylvania," his vision for what he called a "Public Academy of Philadelphia".

However, according to Franklin's autobiography, it was in 1743 when he first had the idea to establish an academy, "thinking the Rev. Richard Peters a fit person to superintend such an institution."

Unlike the other Colonial colleges that existed in 1743—Harvard, William and Mary, and Yale—Franklin's new school would not focus merely on education for the clergy.

He advocated an innovative concept of higher education, one that would teach both the ornamental knowledge of the arts and the practical skills necessary for making a living and doing public service.

The proposed program of study could have become the nation's first modern liberal arts curriculum, although it is never implemented because provost William Smith prefers a traditional curriculum.

Franklin had assembled a board of trustees from among the leading citizens of Philadelphia, the first such non-sectarian board in America.

At the first meeting, held on November 13, 1749, of the twenty-four members of the Board of Trustees, the issue of where to locate the school had been a prime concern.

A  lot across Sixth Street from Independence Hall had been offered without cost by James Logan, its owner, but the Trustees realized that the building erected in 1740, which was still vacant, would be an even better site.

The new board on February 1, 1750, had taken over the building and trusts of the old board.

The Academy of Philadelphia, using the great hall at 4th and Arch Streets, had taken in its first secondary students on August 13, 1751.

A charity school had also been opened in accordance with the intentions of the original "New Building" donors, although it lasted only a few years.

The College of Philadelphia is chartered in 1755, paving the way for the addition of undergraduate instruction.

All three schools share the same Board of Trustees and are considered to be part of the same institution.