Bangladesh spinners Taijul Islam and Mehidy Hasan struck twice to dent New Zealand's reply after the hosts fell to 310 all out on day two of the opening Test on Wednesday.
The tourists reached 78-2 at lunch in Sylhet with Kane Williamson on 26 and Henry Nicholls on 11.
New Zealand bowled out Bangladesh on the first ball of the morning after the hosts resumed on 310-9. Tim Southee trapped last man Shoriful Islam leg-before for 13.
The Black Caps then got off to a fast start, the first three balls yielding 10 runs.
Bangladesh's spinners slowly took control to check the initial batting surge.
Left-arm spinner Taijul dismissed Tom Latham for 21, Nayeem Hasan taking the catch at fine leg.
Fellow opener Devon Conway fell to off-spinner Mehidy two overs later after making 12 runs.
Debutant Shahadat Hossain took a brilliant one-handed catch, diving to his right at silly point after an inside edge hit Conway's pad and popped up to the fielder.
Bangladesh suffered a blow when opening batsman Zakir Hasan had to be stretchered off after being hit by a ball while fielding close in.
The two-match series is the beginning of a new cycle in the World Test Championship for both teams.