The general issue of large $N_c$ QCD at nonzero chemical potential is considered with a focus on understanding the difference between large $N_c$ QCD with an isospin chemical potential and large $N_c$ QCD with a baryon chemical potential. A simple diagrammatic analysis analogous to `t Hooft's analysis at $\mu=0$ implies that the free energy with a given baryon chemical potential is equal to the free energy with an isospin chemical potential of the same value plus $1/N_c$ corrections. Phenomenologically, these two systems behave quite differently. A scenario to explain this difference in light of the diagrammatic analysis is explored. This scenario is based on a phase transition associated with pion condensation when the isospin chemical potential exceeds $m_\pi/2$; associated with this transition there is breakdown of the $1/N_c$ expansion--in the pion condensed phase there is a distinct $1/N_c$ expansion including a larger set of diagrams. While this scenario is natural, there are a number of theoretical issues which at least superficially challenge it. Most of these can be accommodated. However, the behavior of quenched QCD which raises a number of apparently analogous issues cannot be easily understood completely in terms of an analogous scenario. Thus, the overall issue remains open.