The 27-year-old made 21 Championship appearances for the Devon side last season after joining from Blackpool in August.
Grimshaw, who has signed a three-year deal, becomes new Norwich boss Liam Manning's first signing and has played under him previously during a loan spell at Belgian club Lommel SK in the 2020-21 season.
The former Manchester City trainee endured a difficult campaign at Plymouth, including conceding six goals in a hammering at Carrow Road in November, before being replaced by the returning Conor Hazard in January.
"I've known of the interest for a little while and as soon as I heard about it, it was a place I really wanted to come - a big club with big ambitions, so it matched what I wanted to do," Grimshaw told Norwich's club website.