Mass fraction of phosphor in silicone and aging time play important roles in the optics and mechanical performance of the silicone that is used in the light emitting diode (LED) package. In this paper, the mechanical properties of silicone/phosphor composites are investigated experimentally by separate tensile and compression tests. Distribution of the phosphors is observed by scanning electron microscopy (SEM) to ensure the homogeneity of the samples. Different loading rates are applied to study the silicone material's rate-dependent properties. The experimental results of the tensile and compression test show that the Young's modulus increases with the mass fraction of phosphor in silicone. Longer aging time stiffens the silicone composite and weakens the ductility of the materials. A three-dimensional model used cohesive zone material (CZM) between the interface of the phosphor particles, and matrix silicone is built up to study the degradation mechanism at a micro-scale level. The simulation results indicate that the diameter of particles in silicone also impacts its interface debonding and crack growth. The theoretical results concerning the mass fraction of phosphor are in good agreement with the experiments.