We review theoretical models for nonelectromagnetic emission, mainly neutrinos and cosmic rays, from gamma-ray bursts (GRBs). In various stages of the relativistic jet propagation, cosmic-ray ion acceleration and subsequent neutrino emission are expected. GRBs are popular candidate sources of the highest-energy cosmic rays, and their prompt phase has been most widely discussed. IceCube nondetection of PeV neutrinos coincident with GRBs has put interesting constraints on the standard theoretical prediction. The GRB-UHECR hypothesis can critically be tested by future observations. We also emphasize the importance of searches for GeV-TeV neutrinos, which are expected in the precursor/orphan or prompt phase, and lower-energy neutrinos would be more guaranteed and their detections even allow us to probe physics inside a progenitor star. Not only classical GRBs but also low-power GRBs and transrelativistic supernovae can be promising sources of TeV-PeV neutrinos, and we briefly discuss implications for the cumulative neutrino background discovered by IceCube.