The Nikkei share average was down 0.28 per cent at 29,668.74, as of 0202 GMT, while the broader Topix slipped 0.23 per cent to 1,953.80.
“The expectations for the reopening of the economy shrank because rollouts of vaccines in Japan is much slower than other countries, while the number of new COVID-19 cases is on the rise,” said Shoichi Arisawa, general manager of the investment research department at IwaiCosmo Securities.
“The interest rates could fall if the economy slows down. That has sent bank and insurer shares lower on Wednesday.”
Japanese government bond yields tracked overnight U.S. Treasury yields lower on Wednesday.
Vaccinations in Japan have been the slowest among developed economies, while infections are on the rise, and experts worry Tokyo is on the cusp of an “explosive” jump in cases with 100 days to go until Tokyo is scheduled to host the Olympics.
Shipping firms and textiles led declines on the main bourse, with clothing materials maker Toray Industries losing 3.42 per cent.
Department store operator J.Front Retailing dropped 7.42 per cent to became the biggest loser on the Nikkei after flagging a disappointing annual profit forecast.
Insurers fell, with Dai-ichi Life Holdings down 2.13 per cent and T&D Holdings losing 3.89 per cent.
Mega banks also slipped, with Mitsubishi UFJ Financial Group losing 0.46 per cent and Sumitomo Mitsui Financial Group dipping 0.82.
Toshiba jumped 5.77 per cent amid reports of more plans of bids for the conglomerate, though gains were limited after the company announced resignation of Chief Executive Officer Nobuaki Kurumatani.
