Medium-scale quantum devices that integrate about hundreds of physical qubits are likely to be developed in the near future. However, these devices will lack the resources for realizing quantum fault tolerance. Therefore, the main challenge of exploring the advantage of quantum computation is to minimize the impact of device and control imperfections without complete logical encoding. Quantum error mitigation is a solution satisfying the requirement. Here, we demonstrate an error mitigation protocol based on gate set tomography and quasi-probability decomposition. One- and two-qubit circuits are tested on a superconducting device, and computation errors are successfully suppressed. Because this protocol is universal for digital quantum computers and algorithms computing expected values, our results suggest that error mitigation can be an essential component of near-future quantum computation.