Automate screenshots

[alphapapa]

I've made some progress on generating screenshots automatically:

[see second comment]

It uses PhantomJS (which is in Debian). Call it like this:

$ phantomjs --ssl-protocol=any --web-security=false --ignore-ssl-errors=true screenshot.js \
http://www.github.com/alphapapa/solarized-everything-css \
test.png \
../css/solarized-dark/solarized-dark-github.css

Issues that remain:

1. We may need a way to specify a URL other than the domain's front page. For example, the GitHub front page probably isn't what we'd want to display.
2. Since so many sites are forcing HTTPS now, TLS/SSL errors can be a problem, depending on the system. These options seem to fix it, for GitHub at least.

[alphapapa]

@The-Compiler @jgkamat @cal2195 @neeasade @RealyUniqueName

Any of you who are interested, I just pushed a commit to the multi-theme branch that generates screenshots with PhantomJS by running make screenshots. It only generates ones that are out-of-date with respect to the corresponding CSS file. Feel free to give it a try, and please let me know any feedback you have. Thanks.

[jgkamat]

@alphapapa, I think that my pngcrush seems to be different from your version, it seems to error out when I run pngcrush filename.png.

I'm using

 pngcrush 1.7.85, uses libpng 1.6.28 and zlib 1.2.8
 Check http://pmt.sf.net/
 for the most recent version.

The docs for my pngcrush say the usage is similar to

pngcrush [options] [infile.png] [outfile.png]

The error I'm getting is

Traceback (most recent call last):
  File "./make.py", line 239, in <module>
    main()
  File "./make.py", line 33, in main
    update_screenshots()
  File "./make.py", line 96, in update_screenshots
    pool.map(update_screenshot, css_files)
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
subprocess.CalledProcessError: Command '['pngcrush', 'screenshots/darculized/mediawiki.org.png']' returned non-zero exit status 1

Maybe pngcrush should be called with the same infile or outfile? I'm using the pngcrush from the debian repos fwiw.

[alphapapa]

Sorry, I forgot that I had aliased pngcrush on my system to a script that only takes one filename. I'll fix that. Thanks.

[alphapapa]

Done. Should work now!

[jgkamat]

Some of the screenshots seem a bit broken (maybe we need to wait a bit longer for js to load things)
disqus

ddg

But the rest look amazing!

'

It would be nice if we could remove that default no-cookie header from the github one, but I don't think that's worth the effort. Thanks for getting this working, it looks amazing :D

[alphapapa]

Yeah, it could use some more tweaking, for sure.

The Disqus one I'm not sure about. The CSS is probably out-of-date by now, anyway, so maybe we should just remove that sheet. I don't even remember if I've ever used it, myself. What do you think?

The DDG one is weird. We can try adjusting the timeout, but I'm guessing it's either a problem with the CSS we're adding, or something to do with PhantomJS. Or maybe we could load a DDG URL that doesn't use JS?

The GitHub header is really annoying! I think it actually wouldn't be too difficult to fix that: we just need a JS function that clicks to dismiss it, and then we store JS that needs to be run for a screenshot in a file, and load and evaluate that in PhantomJS. I wish we didn't need to do that, but it seems like a reasonable thing to do. Are you good with JS by any chance? I can muddle through it when I have to, but I generally haven't done JS in years, so it takes me a while to google around for things I don't know.

I am reluctant to "pollute" the sites directory with any more non-styl files, though, so I wonder if we should move each site's file into a subdirectory. What do you think?

[jgkamat]

The Disqus one I'm not sure about. The CSS is probably out-of-date by now, anyway, so maybe we should just remove that sheet. I don't even remember if I've ever used it, myself. What do you think?

I don't use disqus either, so I don't mind if it was removed. Maybe we should make a 'retired' folder for retired styl files?

The DDG one is weird. We can try adjusting the timeout, but I'm guessing it's either a problem with the CSS we're adding, or something to do with PhantomJS. Or maybe we could load a DDG URL that doesn't use JS?

Duckduckgo has a nojs mode: https://duckduckgo.com/html (which is probably the best way to do this)

Are you good with JS by any chance?

No, not at all :(.

An easy fix would be to point to the issues page or the readme directly rather than the root of the repo. I don't think it's worth spending effort on trying to clear the header, fwiw :)

I am reluctant to "pollute" the sites directory with any more non-styl files, though, so I wonder if we should move each site's file into a subdirectory. What do you think?

Agreed, maybe we should move supporting files into their own folder, or make a subfolder for every site file (as you suggested)

[alphapapa]

Thanks, I'll see what I can do.

[alphapapa]

I updated the screenshots branch recently, and today @non-Jedi brought to my attention that the GitHub ones are blank. The GitHub stylesheet is actually working fine (I'm using it now), but apparently several sites are no longer compatible with PhantomJS (fewer than were a year ago). I guess we need to use a different headless browser for screenshots.

I'm not interested in working on this anytime soon, so I'm going to un-self-assign and tag it help-wanted, in case someone wanders by who does want to fix it.