At least 78 migrants drowned early on Wednesday and more were feared missing after their overloaded boat capsized and sank off Greece, in one of Europe’s deadliest shipwrecks this year.

By midday, 104 people had been rescued, authorities said, but it remained unclear how many were on board when the vessel -- whose occupants the coast guard said had refused an offer of help late on Tuesday - went under.

"We fear the number of dead will rise," said a shipping ministry official who spoke on condition of anonymity. Greece is one of the main routes into the European Union for refugees and migrants from the Middle East, Asia and Africa.

Most cross over to Greek islands from nearby Turkey, but a growing number of boats also undertake a longer, more dangerous journey from Turkey to Italy via Greece. State broadcaster ERT said the boat that sank had set sail from the Libyan town of Tobruk, which lies south of the Greek island of Crete, and was headed to Italy.

Read more: Pakistanis among 57 dead as migrant boats sink off Libya

The disaster is the deadliest off Greece this year, and among the worst in Europe. In February, 96 people died when their wooden boat smashed into rocks on Italy's Calabrian coast during a storm. Filippo Grandi, the UN High Commissioner for Refugees, called on governments to work together on creating safe pathways for people fleeing poverty and war.

"Only sadness and anger after another deadly tragedy at sea in the Mediterranean," he wrote on Twitter. Greece is currently led by a caretaker government ahead of a national election on June 25. The Greek coast guard said the boat was first spotted late on Tuesday by EU border agency Frontex in international waters around 50 miles (80 km) southwest of the southern Greek coastal town of Pylos.

A Greek coast guard vessel then approached the boat, which was en route to Italy, and offered help. The large number of migrants on its outer deck "refused assistance and stated their desire to continue their voyage," the coast guard said.

A few hours later the boat capsized and sank, triggering a search and rescue operation. Broadcaster ERT said most of those on board were young men in their 20s.

The shipping ministry official said most were from Egypt, Syria and Pakistan, though Greek authorities did not confirm the vessels port of departure. Survivors were taken to the town of Kalamata, where the country's president Katerina Sakellaropoulou was expected to visit.

Greece was at the frontline of Europe's 2015 migration crisis, when nearly 1 million people arrived on its islands from Turkey before heading north to wealthier European states. Numbers have fallen dramatically since a 2016 deal struck between Brussels and Ankara to stem the flows, while the previous conservative government of Kyriakos Mitsotakis had said a tough policy - which included more border patrols and migrant camps under heavy surveillance - helped keep arrivals low.

About 72,000 refugees and migrants have arrived so far this year in Europe's frontline Mediterranean countries, according to United Nations data, with the majority landing in Italy and around 6,500 in Greece. Nearly 1,000 people are estimated to have died or gone missing in the Mediterranean this year, according to the UN.