Request for comments: Adding a GraphQL API

[Søren Løvborg]

For a while, there's been some discussion about Kallithea's API design. It
seems clear that nobody is really happy with the current JSON-RPC API,
which is also in its current form quite limited.

There's been talk about adding a REST-like API, either in a separate API
namespace or in the same namespace as the HTML user interface.

In the latter case, a request like GET /repo/pull-request/42 would return
HTML by default, but JSON if the request came with an "Accept:
application/json" HTTP header. While the idea has a conceptual purity to
it, I'm not convinced by the feasibility of actually having both a
REST-like API and the HTML UI share the same URL namespace. The two are
quite different models, which we've learned the hard way in Kallithea with
its use of _method overrides to fake DELETE and PUT requests from an HTML
form (which can only generate GET and POST), a hack which has caused
numerous headaches. Looking across the web, I know of no large website that
tries to fit both REST API and browser access into the same URL namespace;
they all maintain a separate API namespace.

Even then, deciding to add a new API under e.g. "/_api/" still leaves a lot
of open questions, because the concept of a REST-like API is somewhat
vague. For instance, it's quite straightforward to say "to request a
PullRequest resource by ID, make a GET request to /_api/pull_requests/42",
but a PullRequest consists of a number of "sub-resources", like the PR
owner (a "User" object), reviewers, changesets, comments, etc.

One extreme is to only return IDs (or URLs, if you want to be more
REST-like) for each of these sub-resources. If the caller wants e.g.
changeset details, they'll then have to issue a long series of follow-up
requests to fetch data about each of them, resulting in unacceptable
performance. For this reason, most REST-like APIs return a wide selection
of sub-resource data inline. This, on the other hand, often produces way
too much data, again resulting in bad performance. See e.g. the GitHub API
example for requesting a list of PRs for a repository:
https://developer.github.com/v3/pulls/#response (and note that in that
example, there's only ONE pull request in the resulting list).

At Unity, we've been experimenting with adding GraphQL support to
Kallithea. For those who (like me just a few months ago) haven't heard of
GraphQL, it's a standard for querying structured data (developed by
Facebook), which tries to solve these problems.

Say that I want to get data for a specific PR. I just want the URL, title
and reviewers. I don't need a list of revisions or PR comments; generating
and transmitting those would be a waste of time. On the other hand, I _do_
want the review status for each reviewer, which is a somewhat expensive
operation, and not something we'd want the API endpoint to calculate unless
explicitly requested.

Using GraphQL, I can instead write a query like this, which succinctly
describes the data that I'm after:

    {
        pull_request(id: 58) {
            html_url
            title
            reviewers {
                id
                full_name
                review_status
                avatar(size: 32)
            }
        }
    }

I can URL encode the above and put it into a HTTP URL like this:


https://kallithea-scm.org/_admin/graphql?q=%7Bpull_request%28id%3A58%29%7Bhtml_url%20title%20reviewers%7Bid%20full_name%20review_status%20avatar%28size%3A32%29%7D%7D%7D

(As you can see, GraphQL only has one URL entrypoint, "/_admin/graphql" in
this example.)

The response is plain JSON, for example:

    {
        "data": {
            "pull_request": {
                "html_url": "
https://kallithea-scm.org/repos/kallithea/pull-request/58/_/bump_pytest_to_3.0
",
                "title": "test fixes noticed during Turbogears2 port (v3)",
                "reviewers": [
                    {
                        "id": 3,
                        "full_name": "Mads Kiilerich",
                        "review_status": "not_reviewed",
                        "avatar": "
https://secure.gravatar.com/avatar/c13dec3bbff4e99b50c49c8cdbef6542?d=identicon&s=32
"
                    },
                    {
                        "id": 15,
                        "full_name": "Søren Løvborg",
                        "review_status": "approved",
                        "avatar": "
https://secure.gravatar.com/avatar/170f30ed10c8ec5335aded8181949941?d=identicon&s=32
"
                    }
                ]
            }
        }
    }

While this URL lacks the immediate readability of a more REST-like API, the
original GraphQL query is quite clear, just a URL decode away, and much
more flexible and expressive than anything based on plain key-value
arguments in the URL query string.

I'd like to learn if there's general interest in this approach, and to see
if people feel this would fit their API needs. (Obviously, having a GraphQL
API doesn't preclude us from also having a REST-like API either, but it
might make it redundant.)

My current proof-of-concept is not quite presentable yet, but I hope to
have something to share soon-ish.

More info about GraphQL can be found here: http://graphql.org/

Best,
Søren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.sfconservancy.org/pipermail/kallithea-general/attachments/20160908/1b9a6bf2/attachment.html>