The ventromedial hypothalamus, ventrolateral area (VMHvl) was identified recently as a critical locus for inter-male aggression. Optogenetic stimulation of VMHvl in male mice evokes attack toward conspecifics and inactivation of the region inhibits natural aggression, yet very little is known about its underlying neural activity. To understand its role in promoting aggression, we recorded and analyzed neural activity in the VMHvl in response to a wide range of social and nonsocial stimuli. Although response profiles of VMHvl neurons are complex and heterogeneous, we identified a subpopulation of neurons that respond maximally during investigation and attack of male conspecific mice and during investigation of a source of male mouse urine. These “male responsive” neurons in the VMHvl are tuned to both the inter-male distance and the animal’s velocity during attack. Additionally, VMHvl activity predicts several parameters of future aggressive action, including the latency and duration of the next attack. Linear regression analysis further demonstrates that aggression-specific parameters, such as distance, movement velocity, and attack latency, can model ongoing VMHvl activity fluctuation during inter-male encounters. These results represent the first effort to understand the hypothalamic neural activity during social behaviors using quantitative tools and suggest an important role for the VMHvl in encoding movement, sensory, and motivation-related signals.