Access to emergency food is critical for the survival and health of vulnerable populations, but its importance is not understood in the context of food deserts. Using a cross-sectional survey based on Albany and Troy, New York, we compared the two food desert models, one based on paid (e.g., grocery stores) and the other based on free food options (e.g., emergency food sites such as pantries and soup kitchens). Structural equation modeling was conducted to identify pathways among people’s access to food sites, food consumption patterns, food insecurity, and health conditions. Access to grocery stores did not show significant links to food insecurity or health conditions, whereas access to emergency food, especially time taken to such food outlets, was found to be a significant factor for increased consumption of fresh food. Among the diet-related variables, food insecurity showed the strongest link to negative health outcomes. Access to free or low-cost options needs to be taken into consideration when designing research and practice concerning food deserts, food insecurity, and subsequent health effects.