By losing the opener of a four-game series, the second-place Guardians (67-74) fell 6 1/2 games behind the idle Minnesota Twins in the American League Central.
Cleveland closer Emmanuel Clase, trying for an major-league-best 39th save, instead got his 10th blown save of the season.
Chad Wallach opened the decisive uprising with a one-out double off the right field wall, and Brett Phillips' single to left moved pinch runner Logan O'Hoppe to third.
Rookie Kyren Paris tied the game at 2-2 with a line single to right before the Guardians loaded the bases by intentionally walking Nolan Schanuel. Phillips was gunned down at home on a sharp grounder to Cleveland first baseman Josh Naylor.
Grichuk then lined a single on a 2-2 cutter from Clase, and the Angels celebrated the victory.
Clase (2-8) took the loss. Jose Soriano (1-2) threw one scoreless inning for his first major league as the Angels (65-76) snapped a six-game losing streak.
Naylor's base hit scored Kwan for the first run, and Cleveland made it 2-0 when Ramon Laureano's grounder to short brought in Jose Ramirez.
The Angels cut the lead in half in the seventh when Trey Cabbage scored on Brett Phillips' RBI single to right off former Angel Matt Moore.
Cleveland starter Cal Quantrill scattered three hits and three walks while striking out six over six scoreless innings.
Los Angeles reliever Davis Daniel pitched three scoreless innings on 66 pitches in his major league debut in a bullpen game for the Angels.
The injury-plagued Angels were without outfielder Mickey Moniak, scratched due to back tightness, and slugging DH Shohei Ohtani, who missed a fourth straight game with right oblique tightness.
—Field Level Media