It looks like these guys are picking up ideas from the travis-ci.org project, which uses a ".travis.yml" file for configuration-
https://github.com/tedivm/Stash/blob/master/.travis.yml
This has huge advantages, especially in conjunction with things like git or github. With travis ci it's trivial for testing to be setup on new branches or even forks. We use Jenkins at work, and I'm excited to extend that level of functionality there.
If you don't like the sound of
"The literate plugin adds an Action to all Free-style projects that allows exporting these XML configuration snippets in a .zip file for unpacking into your project's source control."
then you might enjoy CircleCI (https://circleci.com).
We've always believed in understandable configuration checked into source control, with sane defaults and configuration over convention. The configuration (https://circleci.com/docs/configuration) is sane and human readable in yaml, yet incredibly powerful, and allows you set up a simple config in 5 minutes, and a complex config in an hour.
This is based on Don Knuth's Literate Programming: http://en.wikipedia.org/wiki/Literate_programming - but applied to build scripts.
(Readme is docs + executable)
I think one useful feature might be the ability to export existing Jenkins build jobs to the README files. Some ppl might be more comfortable with Jenkins interface rather than writing the README files from scratch.
Congratulations! You have invented the Makefile.
This sounds pointlessly complicated compared to just a shell script with comments that does the same thing.