Issue 33: Entire show doesn't get encoded
Status:  Fixed
Owner:
Closed:  Dec 2008
Reported by smacdiarmid, Nov 13, 2008
What steps will reproduce the problem?
1. Download a show using MPEG-2 (8 Mbps) with comskip
2.
3.

What is the expected output? What do you see instead?
The show downloads entirely, runs comskip completely, but then only partially encodes the file.  
There is no error but the resulting file is only a small portion of the show.  Anywhere from a 
couple of minutes of the show, to about half of it.

What version of the product are you using? On what operating system?
1.4 OS X 10.5.5

Please provide any additional information below.



Nov 13, 2008
Project Member #1 yoav.yer...@gmail.com
Some additional questions:
  Is it only with mpeg-2 (8 mbps) that you only get a few minutes?
  Do you get any messages about mencoder crashing?
  Does the prgoram think that it was a success or does it retry downloading?

  Can you try turning off comskip and turning on 'download first then encode', and
see if it gets the whole show?
Status: Accepted
Owner: yoav.yerushalmi
Nov 13, 2008
Project Member #2 yoav.yer...@gmail.com
Oh one other possibility:
  how much free hard drive space do you have?  It needs to generate a LOT of
temporary data to remove commercials.. you'll need to have at least enough space to
hold the whole show unencoded, plus enough for your encoded show (so at 8mbps, we're
probably talking about 1.5x the size of the show as reported by tivo).  the code
doesn't check to make sure you have the space...
Nov 13, 2008
#3 smacdiarmid
I have had success with mpeg-2 (5 mbps) so it appears to only be a problem with the 8 mbps setting.
There are no messages that mencoder crashes.
The show appears to download successfully (Growl reports the download is complete, and iTivo does not retry 
downloading)

I have plenty of hard drive space, over 500GB free space.

When I get home tonight I will try it turning off comskip and turning on "download first then encode" and 
report back.
Nov 13, 2008
#4 smacdiarmid
I have discovered that it is only a problem with HD shows,  and that it happens with HD shows on both the 
5mbps and the 8mbps setting.

I am currently trying an HD show on MPEG-2 (8mbps) with no comskip and "download first then encode", I will 
post my results
Nov 13, 2008
#5 smacdiarmid
No luck.  The MPEG-2 options both 5mbps and 8mbps do not work with HD shows, even with comskip turned off 
and "download first then encode" selected.

Any ideas?
Nov 14, 2008
Project Member #6 yoav.yer...@gmail.com
Honestly I don't have any useful ideas..

  are the mpeg-4 downloads working ok?  (iphone, quicktime, etc)?  There may be a bug
with the mpeg-2 encoder or the settings I pass to it.  (HD files are pretty big, so
the mpeg-2 encoder might be hitting walls with some frames and the bitrate limits).

  If it's working for an mp4 setting but not mpeg-2, the only thing I can suggest
isn't fun:  Go to the advanced setting after selecting and mpeg-2 format, and start
playing with the video settings. I'm not an expert with the tool either
unfortunately, and documentation about mencoder is very poor.

  Out of interest though:  why are you aiming for mpeg-2 encoding?  the quality is
generally much worse than mpeg4, and an mpeg-2 of an HD movie can't be put on a DVD
or sent back to a regular tivo anyways.  If you just want to use it to edit out
commercials and then tivo go back it,  use 'minimal re-encoding' or whatever it's
called (you won't be able to view it unless you use VLC, but it probably won't barf
out mencoder like the other mpeg-2 settings).

  Yeah, not very helpful   :(
Nov 14, 2008
#7 smacdiarmid
As I have mentioned previously "minimal re-encoding" is totally useless because the audio and video don't sync.  
I guess I will just go back to no encoding and be stuck with the commercials.

Oh well, thanks for trying.
Nov 28, 2008
#8 smacdiarmid
Any new ideas on why HD shows aren't getting encoded entirely?   I have been messing around with some of the 
settings but have not been able to fix it.  It works fine with non-HD shows.

Any thoughts?  Between "minimal re-encoding" having audio and video sync issues, and MPEG-2 not working 
with HD shows, it is impossible to remove commercials and keep the resulting file in a format that works with 
TiVo GoBack.

Thanks.
Nov 29, 2008
#9 smacdiarmid
I seemed to find a fix for it.

I set "vrc_buf_size=18350"

I tried various other values, both lower and higher, and for some reason that is the one that seems to work.  I 
don't know what exactly this does, but I guess it may make sense that the HD shows need a larger buffer to 
encode the show.

Please add an additional setting for MPEG-2 HD that includes this change in the buffer size (and maybe and 
increased bitrate for higher quality).  Thanks.
Nov 29, 2008
Project Member #10 yoav.yer...@gmail.com
Hey!  Thanks for figuring it out.

  I'm changing the 5MBps entry to a 7MBps and calling it 'mpeg-2 HD', and adding the
vrc_buf change to that entry.  It should go out with the next beta.

  Thank you again...
Nov 30, 2008
Project Member #11 yoav.yer...@gmail.com
Ok check out the beta to see if it's doing 'the right thing' for you?
Nov 30, 2008
#12 smacdiarmid
Unfortunately the change in bitrate seems to make the change in buffer size not fix the problem anymore.  I 
have tried increasing the buffer size but it does not seem to fix the problem.

Any ideas on your end?
Nov 30, 2008
Project Member #13 yoav.yer...@gmail.com
ideas no  :(
  Sadly I'm just using an external program that I only understand a little bit of to
do the encoding.  For now, I guess I'll set it back to 5Mbps then (next beta should
get the change).

  In the long run I'm likely to switch to using the handbrake encoder since it is
faster, and maybe that will resolve the problems...
Dec 2, 2008
#14 smacdiarmid
Is it difficult switching to handbrake?  I have found handbrake to be extremely reliable.  It might fix a lot of 
problems and make things much easier in the future.  It sounds like a great idea to me.
Dec 2, 2008
Project Member #15 yoav.yer...@gmail.com
The latest beta supports handbrake.  (still trying things out with it)

 HOWEVER, handbrake doesn't support edit lists, so you can't use it to remove
commercials.
Dec 2, 2008
#16 smacdiarmid
Yeah, I am using the latest beta now, to download and encode in AppleTV format.  I love this option because it 
works equally well with both SD and HD shows (it uses a percentage quality rather than a fixed bitrate).  Adding 
Handbrake options was a great idea.

Is there anyway around the inability to remove commercials?  Or is that never going to happen with Handbrake 
encoding?
Dec 2, 2008
Project Member #17 yoav.yer...@gmail.com
short version:
  to skip commercials, we run comskip on the source mpg, which generates a list of
cut points where the movie needs to be split up. 

  mencoder can take these cutpoints as input while it is encoding, so we use it with
mencoder.

  Handbrake doesn't support cut lists (right now anyways).  So as it is we can't work
around it.  However, it is in theory possible to use mencoder to do a 'dumb'
re-encode which only copies frames over.  However, this will cause a significant
audio-video sync problem.

  So for now, I disable comskip for handbrake.  If/when it becomes possible to do it,
I'll re-enable it.

  I'm going to mark this bug 'fixed' since the 5mbps works for you as mpeg-2 (which
is the point of this bug report).  I see another bug report claiming that the AppleTV
setting isn't working on AppleTV units though (sigh).  So more 'stuff' to figure out
with these darn encoders....
Dec 2, 2008
Project Member #18 yoav.yer...@gmail.com
(No comment was entered for this change.)
Status: Fixed