The ability to generalize previously learned knowledge to novel situations is essential for adaptive behavior. To investigate the neural mechanisms underlying the ability to infer novel compositional word meanings, we developed a behavioral paradigm to probe compositional inference in language. Participants were trained on the meanings of compositional words made up of known stems (“good”) and novel affixes (“kla”) from an artificial language. The meaning of the compositional words depended on the position of the novel affix (“goodkla = bad”, “klahorse = pony”). We then asked participants to infer the meaning of novel compositional words (“whitekla =?”, “klacat =?”) that were either congruent or incongruent with the established rule (“klawhite” is incongruent because a small version of “white” does not exist). During fMRI, participants performed a semantic priming task in which the novel words served as either congruent or incongruent primes (“whitekla”) and their synonyms (“black”) served as targets. Our results demonstrated that participants were able to generate novel compositional meanings on the fly, successfully inferring meanings of congruent versus incongruent words. Univariate analysis of fMRI data at target words revealed a greater repetition suppression effect when primed with congruent than incongruent words in the left inferior frontal gyrus, which suggests that novel meanings are derived at this linguistic “building” hub. Further analysis of congruent versus incongruent prime-related activity revealed a broad frontal-parietal network, including the hippocampus, a brain area commonly associated with the generalization process of structural relationships. Furthermore, we employed multivariate representational similarity analysis to demonstrate structural rule and word meaning representations in hippocampus and left-lateralized core language areas. Together, these findings suggest that compositional generalization in language recruits a domain-general network shared with action planning, compositional vision and constructive relational memory, while newly inferred meanings are represented in more language-specific regions.