This extends the shell of llvm-rc tool with the ability of tokenization
of the input files. Currently, ASCII and ASCII-compatible UTF-8 files
are supported.
Thanks to Nico Weber (thakis) for his original work in this area.
Differential Revision: https://reviews.llvm.org/D35957
llvm-svn: 310621