Although Flash 9.0+ says it supports .mp4 file playback, I’ve found that its actually quite picky about what it will actually play. So, if you want to produce a file using mencoder that plays in flash, you need to:
Download and compile the MP4Box executable from the GPAC distribution
Run an mencoder command to produce an AVI file that has h264 video & AAC audio:mencoder $FILE -o $OUTPUT -oac faac -srate 44100 -ovc x264 -x264encopts bitrate=$BITRATE:turbo:pass=1 -fps 60000/1001 -ofps 30000/1001 -vf field=0,scale=$RES 2>&1 >> $LOG
mencoder $FILE -o $OUTPUT -oac faac -srate 44100 -ovc x264 -x264encopts bitrate=$BITRATE:pass=2 -fps 60000/1001 -ofps 30000/1001 -vf field=0,scale=$RES 2>&1 >> $LOG
Then, you need to pull out the h264 video & AAC audio from the AVI file, and put them back in a mp4 container using MP4Box, like this:
MP4Box -aviraw video $VIDEO_OUTPUT
MP4Box -aviraw audio $AUDIO_OUTPUT
mv $AUDIO_OUTPUT ${AUDIO_OUTPUT/.raw/.aac} # Make the .raw file an .aac file
MP4Box -add $VIDEO_OUTPUT -add $AUDIO_OUTPUT $FINAL_OUTPUT.mp4