Passive radiative cooling technology has attracted extensive attention as it addresses the potential applications in effectively cooling photovoltaics and related systems. Here, we performed comprehensive multidimensional and multiphysical opto-electro-thermal (OET) modeling, which was used to design a silicon-based radiative cooling system for a solar cell (SC). Our study simultaneously takes into account the coupled effects of the radiative cooling characteristics, carrier thermodynamics, and electrodynamic behaviors of SCs in the spatial and frequency domains. Based on a comprehensive photonic design, we presented a radiative cooler with near-ideal spectral selectivity from the sunlight to the infrared band. The fundamental OET physical mechanisms and the effect of temperature on the performance of SCs were explored. A comparable study on the performance parameters of the SCs with and without a radiative cooler was formulated, which revealed that the SC temperature can be reduced by over 10 °C and the absolute power conversion efficiency (PCE) can be increased by 0.45% after employing a photonic radiative cooler. Our OET study provides a ready method to explore the comprehensive OET physics in photovoltaic systems.