Beth Hamedrash Hagadol re-hired Ash to fill the vacant role of congregational rabbi at a salary of $25 per month (or $300—today $7,000—per year).
While rabbi of Beth Hamedrash Hagadol, he founded another yeshiva in Monsey, New York for gifted high school aged boys.
The first Eastern European Orthodox rabbi to serve in the United States, Ash "rejected the reformist tendencies of the German Jewish congregations" there.
The congregation's building at 60–64 Norfolk Street, between Grand Street and Broome Street on the Lower East Side, had originally been the Norfolk Street Baptist Church.
Unlike German Jews, the Jews who founded Beth Hamedrash viewed both religion and the synagogue as central to their lives.
After the court rejected Ash's arguments, a large majority of members left with Ash to form Beth Hamedrash Hagadol ("Great House of Study"), adding the word "Hagadol" ("Great") to the original name.
For the first six years of the congregation's existence, Ash was not paid for his work as rabbi and instead earned a living as a peddler.
In the late 19th century, other synagogues in New York City often served a particular constituency, typically Jews from a single town in Russia, Poland, or Romania.
It built or supported Methodist churches primarily in poor areas, or areas that were being developed, including one in the building that would later house the First Roumanian-American congregation.
Beth Hamedrash Hagadol, the Eldridge Street Synagogue, and 13 other Lower East Side synagogues had raised $2,500 (today $65,000) towards the creation of a European style kehilla to oversee New York's Orthodox community, and had imported Joseph in an attempt to achieve that (ultimately unfulfilled) goal.
