Local authorities said the vessel capsized when another large tour ship passed by the smaller boat, creating a wake that caused the accident.
The 12-metre, glass-enclosed Ethan Allen tour boat was carrying elderly Canadian citizens when it sank.
“The unofficial number right now that we have on board is 49 people,” said Warren County Sheriff Larry Cleveland, speaking to AFP.
“Of those 49 people, we do know that 21 people have perished,” he said.
Television footage has showed white body bags lined up on the grassy shore of the lake, amid picturesque hills under a clear blue sky.
The accident occurred in a popular resort area about 320 kilometres north of New York city and about 80 kilometres north of Albany in the Adirondack Mountains.
The boat had the capacity to hold around 50 people, according to media reports.
The weather was clear, calm and in the low 20s Celsius and witnesses said the accident appears to have not been weather-related.
Representatives of Shoreline Cruises, which operates the boat, could not immediately be reached for comment.
“I saw like 20 people that were laying on the grass that were covered. And they were dead,” witness Andrea Sause told CNN.
She had been cleaning rooms in a nearby building when she heard ambulances and went to investigate with her father.
“We were helping, bringing blankets, and we were pulling the elderly out of the boat,” she said.
“They were all frantic, and a lot of them had chest problems.”
Another witness said a number of people were trapped under the boat when it tipped over.
“They did try to lift up the boat,” said Joanne Rahal.
“But then, when they did that, the boat started to sink all the way … There were people floating in the water.”
Firefighers, paramedics and police were at the scene, which occurred around 3pm local time (5am AEST).