We construct a Lagrangian formulation of N = 4 supersymmetric mechanics with hyper-Kähler sigma models in a bosonic sector in a non-Abelian background gauge field. The resulting action includes a wide class of N = 4 supersymmetric mechanics describing the motion of an isospin-carrying particle over spaces with non-trivial geometry. In two examples that we discuss in details, the background fields are identified with the field of BPST instantons in flat and Taub-NUT
spaces.