In this paper, we report an easy route for preparing new metal nanorod-polymer composites consisting of gold nanorods, Au NRs, and temperature responsive copolymer “microgel” particles. The microgel particles of ~200 nm in size, which contain carboxylic acid groups, were prepared by surfactant-free emulsion polymerization of a selected mixture made of N-isopropylacylamide and acrylic acid in the presence of a cross-linker N,N′-methylenebisacrylamide. The electrostatic interactions between the cationic cetyltrimethylammonium bromide (CTAB) stabilized Au NRs and anionic microgel particles were expected to occur in order to prepare stable Au NRs-microgel composite particles. The optical and structural characterization of the composite was achieved using UV-Vis spectroscopy, Field emission scanning electron microscopy (FESEM), Transmission electron microscopy (TEM) and dynamic light scattering (DLS). TEM image shows that Au NRs are attached on the surface of the microgel particles. Dynamic light scattering measurements prove that the composite particles are temperature responsive, which means the particles undergo a decrease in size as the temperature increases above its phase transition temperature. In vitro cytotoxicity of the composite materials were tested by 3-(4,5-Dimethylthiazol-2-yl)-2,5-diphenyltetrazolium bromide (MTT), Lactate dehydrogenase (LDH), and hemolysis assay, which showed non-toxicity (biocompatibility).