Despite interest in addressing social determinants of health to improve patient outcomes, little progress has been made in integrating social services with medical care. We aimed to understand how health care providers with strong motivation (for example, operating under new payment models) and commitment (for example, early adopters) fared at addressing patients' social needs. We collected qualitative data from twenty-two accountable care organizations (ACOs). These ACOs were early adopters and were working on initiatives to address social needs, including such common needs as transportation, housing, and food. However, even these ACOs faced significant difficulties in integrating social services with medical care. First, the ACOs were frequently "flying blind," lacking data on both their patients' social needs and the capabilities of potential community partners. Additionally, partnerships between ACOs and community-based organizations were critical but were only in the early stages of development. Innovation was constrained by ACOs' difficulties in determining how best to approach return on investment, given shorter funding cycles and longer time horizons to see returns on social determinants investments. Policies that could facilitate the integration of social determinants include providing sustainable funding, implementing local and regional networking initiatives to facilitate partnership development, and developing standardized data on community-based organizations' services and quality to aid providers that seek partners.