When the fuel pump was replaced, was the fuel pressure regulator that sits on top of the fuel pump also replaced? The fuel pressure regulator contains a check valve that keeps fuel in the fuel rail next to the fuel injectors for a fast start.
Try this... try turning the ignition key to the On position 4-5 times to cycle the fuel pump enough so it can refill the fuel line if a leaky check valve has allowed it to drain back down into the fuel tank. If the engine starts up faster after doing that, something in the fuel system... check valve, fuel injector, etc. is leaking and allowing the gasoline that is supposed to stay put in the fuel rail for a fast start to instead drain out and return back to the fuel tank instead. The delay in starting is normally the amount of time it takes the fuel pump to get fuel back up to the fuel injectors again.