The Markit/Nikkei services Purchasing Managers' Index (PMI) for Japan decreased to 50.0 in March from 51.2 in February.
A reading below 50 indicates contraction of activity, while a reading above 50 indicates expansion.
The decrease was driven by a decline in employment.
"General business conditions in the Japanese service sector broadly stabilised in March. Both activity and new orders were unchanged from February, when only modest increases were seen. As a result, services firms were less optimistic towards their hiring policies, with job numbers decreasing for the first time since last November," economist at Markit, Amy Brownbill, said.