Yes it will reset itself if the gas cap fixed the leak. Probably a day or two at the most. It is common for locking gas caps to leak enough air to cause the CEL.
To immediately turn off the CEL, if everything is leak free now, you can also either disconnect the battery for two miinutes or disconnect the three connectors on the PCM for two minutes. Doing the latter instead of disco'ing the battery will preserve your radio's clock setting and station presets.