Skip to content

[transcript] Only generate HTMLs when the transcript is modified or new

David Revoy requested to merge transcript-improvements into master

Re-rendering then re-uploading all the HTMLs happens all the time and the process takes between 10 to 20min now. It's a time for me lost and each MR now asks (more or less) for re-running validate-transcripts.md...

So, while it's ok to keep doing the maintainance of MR this way (I sort of adapted to the workflow); I decided to study how to cache things already rendered so I'll not have to wait the long upload to the distant server of all htmls for all episode in all langs again and again.

This MR creates a 'cache' directory in each episode directory and create a markdown copy of the transcript in it. Then at the next loop, it compares the transcript markdown with the previous cached version. If the two differs; it renders the HTML files. This is much more economic in terms of uploading bandwith for later upload of the only generated htmls; and for the local disk (SSD wearing?) too.

This will also stop the noise in the uploader about thousands of HTML files transiting each time that hides me what are the real changes that are uploaded to the server (hard to spot 10 files rendered in the middle of thousands of html transiting).

Question to @gunchleoc before merging this MR: (First, happy new year! 🎉) In this change, I'm pluging the CLI command diff to the validate-transcripts.sh script. I know last time I plugged a new command ( it was parallel) it required a special modification in the CI/CD to avoid an error, like, installing the package. The question: will it be the case also for diff too, or is it safe to merge this modification as it is?

Thanks!

Merge request reports