We present mid-infrared observations of active galactic nuclei (AGNs) in the GOODS fields, performed with the Spitzer Space Telescope. These are the deepest infrared and X-ray fields to date and cover a total area of similar to 0.1 deg(2). AGNs are selected on the basis of their hard (2-8 keV) X-ray emission. The median AGN infrared luminosity is at least 10 times larger than the median for normal galaxies with the same redshift distribution, suggesting that the infrared emission is dominated by the central nucleus. The X-ray-to-infrared luminosity ratios of GOODS AGNs, most of which are at 0.5 less than or similar to z less than or similar to 1.5, are similar to the values obtained for AGNs in the local universe. The observed infrared flux distribution has an integral slope of similar to 1.5, and there are 1000 sources per square degree brighter than similar to 50 mu Jy at similar to 3-6 mu m. The counts approximately match the predictions of models based on AGN unification, in which the majority of AGNs are obscured. This agreement confirms that the faintest X-ray sources, which are dominated by the host galaxy light in the optical, are obscured AGNs. Using these Spitzer data, the AGN contribution to the extragalactic infrared background light is calculated by correlating the X-ray and infrared catalogs. This is likely to be a lower limit given that the most obscured AGNs are missed in X-rays. We estimate the contribution of AGNs missed in X-rays, using a population synthesis model, to be similar to 45% of the observed AGN contribution, making the AGN contribution to the infrared background at most similar to 2%-10% in the 3-24 mu m range, depending on wavelength, lower than most previous estimates. The AGN contribution to the infrared background remains roughly constant with source flux in the IRAC bands but decreases with decreasing flux in the MIPS 24 mu m band, where the galaxy population becomes more important.