Table Of Contents

Previous topic

Quickstart

Next topic

RBR specification

This Page

Full example: an URL shortener

Warning

We make the assumption here that you are familiar with WebOb and Routes

You can find a more complex version of this demo , and other demos at: http://bitbucket.org/tarek/redbarrel/src/tip/redbarrel/demos

Let’s create an URL shortener with RedBarrel. The shortener has the following features:

  • an api to generate a shortened URL, given an URL
  • a redirect to the target URL
  • an api to get visit statistics for every redirect
  • an HTML page with a form to create a shortener

There’s no authentication at all for the sake of simplicity.

Step 1: the RBR file

Let’s create a RBR file with the rbr-quickstart command:

$ rbr-quickstart
Name of the project: ShortMe
Description: An URL Shortener
Version [1.0]:
Home page of the project: http://example.com
Author: Tarek
Author e-mail: tarek@ziade.org
App generated. Run your app with "rbr-run shortme.rbr"

The RBR file produced provides a simple hello world API:

# generated by RedBarrel's rbr-quickstart script
meta (
    description """An URL Shortener""",
    version 1.0
);

path hello (
    description "Simplest service",
    method GET,
    url /,
    response-body (
        description "Returns an Hello word text"
    ),
    response-headers (
        set Content-Type "text/plain"
    ),
    use python:shortme.hello.hello
);

Let’s remove the hello definition and add ours. We want to add:

  • a POST for the URL shortener
  • a GET for the stats
  • the redirect
  • the HTML page and its corresponding action

Let’s describe the POST first. We want the URL to shorten in the request body, and get the result back in the response body:

path shorten (
    description "Shortens an URL",
    method POST,
    url /shorten,
    request-body (
        description "Contains the URL to shorten"
    ),
    response-body (
        description "The shortened URL"
    ),
    response-headers (
        set Content-Type "text/plain"
    ),
    response-status (
        describe 200 "Success",
        describe 400 "I don't like the URL provided"
    ),

    use python:shortme.shorten.shorten
);

The description is quite simple:

  • the service is mapped at /shorten
  • the url to be shortened is provided in a POST body
  • the response is a plain text with the shortened url
  • the server returns a 200 or a 400
  • the code will be located in the shorten() function in the shorten module in the shortme package.

The way the shortener works is:

  1. a unique id is created on the server for every new URL, and kept in a dict
  2. /shorten returns a URL that looks like : /r/ID
  3. When the redirect API is called, the server looks for the URL in the dict and redirects to it with a 303. In case it’s not present in the dict, a 404 is returned.

The redirect is expressed as follows:

path redirect (
    description "Redirects",
    method GET,
    url /r/{redirect},
    response-status (
        describe 303 "Redirecting to original",
        describe 404 "Not Found"
    ),
    use python:shortme.shorten.redirect
);

The stats service computes statistics and returns them in json:

path stats (
    description "Returns a number of hits per redirects",
    method GET,
    url /stats,
    response-status (
        describe 200 "OK",
        describe 503 "Something went wrong"
    ),
    response-body (
        description "A mapping of url/hits",
        unless type is json return 503
    ),
    response-headers (
        set Content-Type "application/json"
    ),
    use python:shortme.shorten.stats
);

Stats are just simple counters for every URL, that get incremented everytime a redirect is done.

For the HTML page that let people create shortened URL, we return a page created with a template:

path shorten_html (
    description "HTML view to shorten an URL",
    method GET,
    url /shorten.html,
    response-headers (
        set Content-Type "text/html"
    ),
    response-status (
        describe 200 "Success"
    ),
    use python:shortme.shorten.shorten_html
);

Last, a second page is displayed for the shortening result:

path shortened_html (
    description "HTML page that displays the result",
    method GET,
    url /shortened.html,
    response-headers (
        set Content-Type "text/html"
    ),
    response-status (
        describe 200 "Success"
    ),
    use python:shortme.shorten.shortened_html
);

Let’s save the file then verify its syntax:

$ rbr-check shortme.rbr
Syntax OK.

The syntax checker just controls that your file is RBR compliant, by parsing it. It’s useful to catch any syntax error, like a missing comma.

Notice that the checker does not check that:

  1. the code and file locations are valid
  2. there are duplicate definitions

Those are checked when the application gets initialized, and will generate errors.

Step 2: the code

Let’s create the code now !

Now we can add a shorten module in our shortme package and add our functions in it. We have simple functions for this application but you can use classes or whatever you want to organize your application.

RedBarrel does not impose anything here.

The shorten() function gets the URL to shorten in the request’s POST or body, depending if it was called directly or via the HTML form:

from webob.exc import HTTPNotFound, HTTPSeeOther

_SHORT = {}
_DOMAIN = 'http://localhost:8000/'

def shorten(globs, request):
    if 'url' in request.POST:
        # form
        url = request.POST['url']
        redirect = True
    else:
        # direct API call
        url = request.body
        # no redirect
        redirect = False

    next = len(_SHORT)
    if url not in _SHORT:
        _SHORT[next] = url

    shorten = '%sr/%d' % (_DOMAIN, next)

    if not redirect:
        return shorten
    else:
        location = '/shortened.html?url=%s&shorten=%s' \
                % (url, shorten)
        raise HTTPSeeOther(location=location)

Warning

This is a toy implementation. Do not run a shortener with this code ;-)

If the call was made from the html page, the API redirects to the result HTML page, otherwise it returns the shortened URL.

Once the URL is created, redirect() may be called via /r/someid:

_HITS = defaultdict(int)

def redirect(globs, request):
    """Redirects to the real URL"""
    path = request.path_info.split('/r/')
    if len(path) < 2 or int(path[-1]) not in _SHORT:
        raise HTTPNotFound()
    index = int(path[-1])
    location = _SHORT[index]
    _HITS[index] += 1
    raise HTTPSeeOther(location=location)

Every call increments a hit counter, that’s used in stats():

def stats(globs, request):
    """Returns the number of hits per redirect"""
    res = [(url, _HITS[index]) for index, url in _SHORT.items()]
    return json.dumps(dict(res))

Last, the two HTML pages are simple string templates:

def shortened_html(globs, request, url='', shorten=''):
    """HTML page with the shortened URL result"""
    tmpl = os.path.join(os.path.dirname(__file__), 'shortened.html')
    with open(tmpl) as f:
        tmpl = f.read()
    return tmpl % {'url': url, 'shorten': shorten}


def shorten_html(globs, request, url=''):
    """HTML page to create a shortened URL"""
    tmpl = os.path.join(os.path.dirname(__file__), 'shorten.html')
    with open(tmpl) as f:
        tmpl = f.read()
    return tmpl % url

Notice that the GET params are passed through keywords arguments.

That’s it !

To run the application, just execute the RBR file with rbr-run:

$ ../bin/rbr-run shortme.rbr
Generating LALR tables
Initializing the globs...
Generating the Web App...
=> 'shorten' hooked for '/shorten'
=> 'shorten_html' hooked for '/shorten.html'
=> 'shortened_html' hooked for '/shortened.html'
=> 'redirect' hooked for '/r/{redirect}'
=> 'stats' hooked for '/stats'
...ready

Serving on port 8000...

You can visit the API documentation at /__doc__, which is generated automatically for you.

Step 3: release & deploy

To release and deploy applications, RedBarrel uses the existing standards:

  • distutils
  • WSGI

If you are familiar with those, this section should not be surprising.

Creating releases

The wizard creates a setup.py file and a setup.cfg file, you can use to create a release with Distutils.

With Distutils1:

$ python setup.py sdist

With Distutils2:

$ pysetup run sdist

The setup.py file is just a wrapper around the setup.cfg file so distutils-based installers are made happy.

Running behind a Web Server

Running the application via rbr-run is just for development and tests usage. In production, we want to use a real Web Server like Nginx or Apache.

Since RedBarrel produces a WSGI application, it’s very easy to provide a script for mod_wsgi or Gunicorn or any WSGI-compatible server. There’s one default wsgiapp.py script provided when you run the wizard, located in the package created:

# generated by RedBarrel's rbr-quickstart
from redbarrel.wsgiapp import WebApp
application = WebApp('shortme.rbr')

Running it with GUnicorn is as simple as:

$ gunicorn shortme.wsgiapp
2011-07-15 14:50:42 [14316] [INFO] Starting gunicorn 0.11.2
2011-07-15 14:50:42 [14316] [INFO] Listening at: http://127.0.0.1:8000 (14316)
2011-07-15 14:50:42 [14319] [INFO] Booting worker with pid: 14319
Initializing the globs...
Generating the Web App...
=> 'index' hooked for '/'
=> 'shorten' hooked for '/shorten'
=> 'shorten_html' hooked for '/shorten.html'
=> 'shortened_html' hooked for '/shortened.html'
=> 'redirect' hooked for '/r/{redirect}'
=> 'stats' hooked for '/stats'
...ready

Congrats, you now have a RedBarrel app that scales ;)