There are great challenges in developing efficient adsorbents to replace the currently used and energy-intensive cryogenic distillation processes for olefin/paraffin separation, owing to the similar physical properties of the two molecules. Here we report an ultramicroporous metal-organic framework [Ca(CO)(HO)], synthesized from calcium nitrate and squaric acid, that possesses rigid one-dimensional channels. These apertures are of a similar size to ethylene molecules, but owing to the size, shape and rigidity of the pores, act as molecular sieves to prevent the transport of ethane. The efficiency of this molecular sieve for the separation of ethylene/ethane mixtures is validated by breakthrough experiments with high ethylene productivity under ambient conditions. This material can be easily synthesized at the kilogram scale using an environmentally friendly method and is water-stable, which is important for potential industrial implementation. The strategy of using highly rigid metal-organic frameworks with well defined and rigid pores could also be extended to other porous materials for chemical separation processes.