pexels-photo-3568518.jpeg

In part 1 we designed our API using OpenAPI/​Swagger. Now it’s time to write some tests and wire it up using Mojolicious::Plugin::OpenAPI. This is a much longer article; buckle up!

If you haven’t already, you’ll need to install Perl. Using Linux or macOS? You’ve already got Perl installed on your system. On Windows? I recommend you install Strawberry Perl as it lets you develop Perl applications using the same tools that our Unix-​based brethren use. 

(Advanced users may want to investigate using perlbrew, plenv, or berrybrew for managing multiple versions of Perl and installing more recent versions than are included on your system.)

Once you have Perl installed, it’s a simple matter of using either the cpan or cpanm tools to install Mojolicious and Mojolicious::Plugin::OpenAPI. The latter will install some dependencies as well. Here’s a transcript of installing cpanm and the two modules:

% curl -L https://cpanmin.us | perl - App::cpanminus
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  295k  100  295k    0     0  97765      0  0:00:03  0:00:03 --:--:-- 97765
--> Working on App::cpanminus
Fetching http://www.cpan.org/authors/id/M/MI/MIYAGAWA/App-cpanminus-1.7044.tar.gz ... OK
Configuring App-cpanminus-1.7044 ... OK
Building and testing App-cpanminus-1.7044 ... OK
Successfully installed App-cpanminus-1.7044
1 distribution installed
% cpanm Mojolicious Mojolicious::Plugin::OpenAPI
--> Working on Mojolicious
Fetching http://www.cpan.org/authors/id/S/SR/SRI/Mojolicious-9.01.tar.gz ... OK
Configuring Mojolicious-9.01 ... OK
Building and testing Mojolicious-9.01 ... OK
Successfully installed Mojolicious-9.01
--> Working on Mojolicious::Plugin::OpenAPI
Fetching http://www.cpan.org/authors/id/J/JH/JHTHORSEN/Mojolicious-Plugin-OpenAPI-4.00.tar.gz ... OK
Configuring Mojolicious-Plugin-OpenAPI-4.00 ... OK
==> Found dependencies: JSON::Validator
--> Working on JSON::Validator
Fetching http://www.cpan.org/authors/id/J/JH/JHTHORSEN/JSON-Validator-4.14.tar.gz ... OK
Configuring JSON-Validator-4.14 ... OK
==> Found dependencies: YAML::PP, Test::Deep
--> Working on YAML::PP
Fetching http://www.cpan.org/authors/id/T/TI/TINITA/YAML-PP-0.026.tar.gz ... OK
Configuring YAML-PP-0.026 ... OK
==> Found dependencies: Test::Warn, Test::Deep
--> Working on Test::Warn
Fetching http://www.cpan.org/authors/id/B/BI/BIGJ/Test-Warn-0.36.tar.gz ... OK
Configuring Test-Warn-0.36 ... OK
==> Found dependencies: Sub::Uplevel
--> Working on Sub::Uplevel
Fetching http://www.cpan.org/authors/id/D/DA/DAGOLDEN/Sub-Uplevel-0.2800.tar.gz ... OK
Configuring Sub-Uplevel-0.2800 ... OK
Building and testing Sub-Uplevel-0.2800 ... OK
Successfully installed Sub-Uplevel-0.2800
Building and testing Test-Warn-0.36 ... OK
Successfully installed Test-Warn-0.36
--> Working on Test::Deep
Fetching http://www.cpan.org/authors/id/R/RJ/RJBS/Test-Deep-1.130.tar.gz ... OK
Configuring Test-Deep-1.130 ... OK
Building and testing Test-Deep-1.130 ... OK
Successfully installed Test-Deep-1.130
Building and testing YAML-PP-0.026 ... OK
Successfully installed YAML-PP-0.026
Building and testing JSON-Validator-4.14 ... OK
Successfully installed JSON-Validator-4.14
Building and testing Mojolicious-Plugin-OpenAPI-4.00 ... OK
Successfully installed Mojolicious-Plugin-OpenAPI-4.00
7 distributions installed

(The versions listed above may differ as this article gets progressively out of date.)

Next, make a folder somewhere to place our new microservice project. We’ll call it ~/Projects/blog here, but any place will do as long as you know how to get to it from your text editor and your command line.

After that, go into that directory and use the newly-​installed mojo command to build the basics of our project:

% cd ~/Projects/blog
% mojo generate app Local::Dictionary::Microservice
  [mkdir] /Users/mgardner/Projects/blog/local_dictionary_microservice/script
  [write] /Users/mgardner/Projects/blog/local_dictionary_microservice/script/local_dictionary_microservice
  [chmod] /Users/mgardner/Projects/blog/local_dictionary_microservice/script/local_dictionary_microservice 744
  [mkdir] /Users/mgardner/Projects/blog/local_dictionary_microservice/lib/Local/Dictionary
  [write] /Users/mgardner/Projects/blog/local_dictionary_microservice/lib/Local/Dictionary/Microservice.pm
  [exist] /Users/mgardner/Projects/blog/local_dictionary_microservice
  [write] /Users/mgardner/Projects/blog/local_dictionary_microservice/local-dictionary-microservice.yml
  [mkdir] /Users/mgardner/Projects/blog/local_dictionary_microservice/lib/Local/Dictionary/Microservice/Controller
  [write] /Users/mgardner/Projects/blog/local_dictionary_microservice/lib/Local/Dictionary/Microservice/Controller/Example.pm
  [mkdir] /Users/mgardner/Projects/blog/local_dictionary_microservice/t
  [write] /Users/mgardner/Projects/blog/local_dictionary_microservice/t/basic.t
  [mkdir] /Users/mgardner/Projects/blog/local_dictionary_microservice/public
  [write] /Users/mgardner/Projects/blog/local_dictionary_microservice/public/index.html
  [mkdir] /Users/mgardner/Projects/local_dictionary_microservice/templates/layouts
  [write] /Users/mgardner/Projects/local_dictionary_microservice/templates/layouts/default.html.ep
  [mkdir] /Users/mgardner/Projects/blog/local_dictionary_microservice/templates/example
  [write] /Users/mgardner/Projects/blog/local_dictionary_microservice/templates/example/welcome.html.ep

This command builds the scaffolding for our new project. We won’t be using all of it, but it does provide some useful starting points.

(Why did we put Local:: in the beginning of the name? Because nothing you place there will conflict with other modules you install from CPAN.)

Next, put the OpenAPI document we developed in part 1 somewhere in the project. We’ll choose openapi/dictionary_openapi.yml. Create a folder called openapi in the project, and then open your text editor and paste the following document into a file called dictionary_openapi.yml.

openapi: 3.0.3
info:
  title: Dictionary
  description: The PhoenixTrap.com dictionary microservice
  version: 1.0.0
  license:
    name: Artistic License 2.0
    url: https://www.perlfoundation.org/artistic-license-20.html
paths:
  /health:
    get:
      summary: Check if this service is online
      x-mojo-to: monitoring#heartbeat
      responses:
        200:
          description: All systems operational
          content:
            application/json:
              schema:
                type: object
        500:
          description: Something is wrong
  /word/{word}:
    get:
      summary: Get the definition of a word
      x-mojo-to: word#define
      parameters:
      - $ref: '#/components/parameters/word'
      responses:
        200:
          description: Found word
          content:
            application/json:
              schema:
                type: string
        404:
          description: Could not find word
    post:
      summary: Add or replace the definition of a word
      x-mojo-to: word#save
      parameters:
      - $ref: '#/components/parameters/word'
      requestBody:
        description: Definition of a word
        required: true
        content:
          application/x-www-form-urlencoded:
            schema:
              type: object
              properties:
                definition:
                  type: string
      responses:
        200:
          description: Word saved
    delete:
      summary: Delete an entry from the dictionary
      x-mojo-to: word#remove
      parameters:
      - $ref: '#/components/parameters/word'
      responses:
        200:
          description: Word deleted
components:
  parameters:
    word:
      description: A word in the dictionary
      in: path
      name: word
      required: true
      schema:
        type: string

Calling the plugin

Right now all our application does is serve a demonstration page; you can test it by running the following command:

% script/local_dictionary_microservice daemon
[2021-02-28 16:17:09.76863] [74167] [info] Listening at "http://*:3000"
Web application available at http://127.0.0.1:3000

…and then going to that URL on the last line.

After you’ve verified that it works, hold down the Control key and type the letter C in your terminal to stop the script. Edit the lib/Local/Dictionary/Microservice.pm file in your text editor. Replace it with this:

package Local::Dictionary::Microservice;
use Mojo::Base 'Mojolicious', -signatures;

# This method will run once at server start
sub startup($self) {

    # Load configuration from config file
    my $config = $self->plugin('NotYAMLConfig');

    # Configure the application
    $self->secrets( $config->{secrets} );
    $self->plugin( OpenAPI => {
        url => $self->home
          ->rel_file('openapi/dictionary_openapi.yml'),
    } );
}

1;

Now when you run the script as a daemon, the URL it reports responds with the JSON version of our OpenAPI document. Progress!

Note the $self->plugin() call towards the end of our class above. That’s the secret sauce that loads our OpenAPI document and tells Mojolicious to create responses from it.

Writing our first tests

Next, we’ll write a couple of test scripts. (Why are we writing tests before actually coding our microservice? Because we’re practicing test-​driven development, in which we write our tests first, see that they fail, and then write the code to make them succeed.) Here’s a modified t/basic.t script that should succeed right off the bat:

use Mojo::Base -strict;

use Test::More;
use Test::Mojo;

my $t = Test::Mojo->new('Local::Dictionary::Microservice');
$t->get_ok('/')->status_is(200)->json_has('/openapi');

done_testing();

Run it with prove:

% prove -vl t/basic.t
t/basic.t .. [2021-02-28 16:32:54.79287] [74611] [debug] [3kzjyqgi] GET "/"
[2021-02-28 16:32:54.79321] [74611] [debug] [3kzjyqgi] Routing to a callback
[2021-02-28 16:32:54.79536] [74611] [debug] [3kzjyqgi] 200 OK (0.002493s, 401.123/s)
ok 1 - GET /
ok 2 - 200 OK
ok 3 - has value for JSON Pointer "/openapi"
1..3
ok
All tests successful.
Files=1, Tests=3,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.45 cusr  0.07 csys =  0.55 CPU)
Result: PASS

Now for a test that will fail at first. Put this in t/word.t:

use Mojo::Base -strict;

use Test::More;
use Test::Mojo;

my $t = Test::Mojo->new('Local::Dictionary::Microservice');

$t->post_ok(
    '/word/foo' =>
      form => { definition => 'A metasyntactic variable' },
)->status_is(200);

done_testing();

And prove it:

% prove -vl t/word.t
t/word.t .. [2021-02-28 16:48:53.26826] [75751] [debug] [SlDSHWVE] POST "/word/foo"
[2021-02-28 16:48:53.26897] [75751] [debug] [SlDSHWVE] Routing to controller "Local::Dictionary::Microservice::Controller::Word" and action "save"
[2021-02-28 16:48:53.27278] [75751] [debug] [SlDSHWVE] Template "word/save.html.ep" not found
[2021-02-28 16:48:53.27287] [75751] [debug] [SlDSHWVE] Nothing has been rendered, expecting delayed response
[2021-02-28 16:49:23.27553] [75751] [debug] Inactivity timeout
# Premature connection close
not ok 1 - POST /word/foo
not ok 2 - 200 OK
#   Failed test 'POST /word/foo'
#   at t/word.t line 8.
#   Failed test '200 OK'
#   at t/word.t line 8.
#          got: undef
#     expected: '200'
1..2
# Looks like you failed 2 tests of 2.
Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/2 subtests
Test Summary Report
-------------------
t/word.t (Wstat: 512 Tests: 2 Failed: 2)
  Failed tests:  1-2
  Non-zero exit status: 2
Files=1, Tests=2, 31 wallclock secs ( 0.02 usr  0.01 sys +  0.45 cusr  0.07 csys =  0.55 CPU)
Result: FAIL

Note in the second line of output that Mojolicious tried to route to a controller class at Local::Dictionary::Microservice::Controller::Word with an action of save. That’s what our OpenAPI document told it to do with its x-mojo-to: word#save line.

Adding methods to make the tests pass

Now we’ll write the controller class at lib/Local/Dictionary/Microservice/Controller/Word.pm:

package Local::Dictionary::Microservice::Controller::Word;
use Mojo::Base 'Mojolicious::Controller', -signatures;
use DB_File;

sub save($self) {
    return if not $self->openapi->valid_input;
    my ( $word, $definition ) = map { $self->param($_) }
      qw(word definition);

    tie my %definition, 'DB_File',
      $self->app->home->child('definitions.db');
    $definition{$word} = $definition;

    return $self->render( openapi => {} );
}

1;

The save method above retrieves the word and definition parameters from the URL path and POST data, respectively, and then saves them into a hash that is tied to a Berkeley DB file stored in definitions.db. It then renders an empty response back to the client.

Now run the test:

% prove -vl t/word.t
t/word.t .. [2021-02-28 18:06:33.77551] [78923] [debug] [VV0L-5rU] POST "/word/foo"
[2021-02-28 18:06:33.77623] [78923] [debug] [VV0L-5rU] Routing to controller "Local::Dictionary::Microservice::Controller::Word" and action "save"
[2021-02-28 18:06:33.77751] [78923] [debug] [VV0L-5rU] 200 OK (0.001994s, 501.505/s)
ok 1 - POST /word/foo
ok 2 - 200 OK
1..2
ok
All tests successful.
Files=1, Tests=2,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.43 cusr  0.07 csys =  0.53 CPU)
Result: PASS

Woohoo, our method works! Let’s add some tests to the same t/word.t script, underneath the first one:


$t->get_ok('/word/foo')->status_is(200)
  ->content_is('"A metasyntactic variable"');

$t->delete_ok('/word/foo')->status_is(200);

$t->get_ok('/word/foo')->status_is(404);

Here we’re testing that we get back the definition we saved, that we can delete the definition, and that when we try to retrieve it again we get a 404 Not Found error.

The test script will fail again, but we can fix that by adding the define and remove methods to our controller class in lib/Local/Dictionary/Microservice/Controller/Word.pm:

sub define($self) {
    return if not $self->openapi->valid_input;
    my $word = $self->param('word');

    tie my %definition, 'DB_File',
      $self->app->home->child('definitions.db');

    return $self->render( openapi => $definition{$word},
      exists $definition{$word} ? () : (status => 404),
    );
}

sub remove($self) {
    return if not $self->openapi->valid_input;
    my $word = $self->param('word');

    tie my %definition, 'DB_File',
      $self->app->home->child('definitions.db');
    delete $definition{$word};

    return $self->render( openapi => {} );
}

Run the test script one last time:

% prove -vl t/word.t
t/word.t .. [2021-02-28 18:13:50.82904] [79162] [debug] [7PprxDOz] POST "/word/foo"
[2021-02-28 18:13:50.82978] [79162] [debug] [7PprxDOz] Routing to controller "Local::Dictionary::Microservice::Controller::Word" and action "save"
[2021-02-28 18:13:50.83225] [79162] [debug] [7PprxDOz] 200 OK (0.003191s, 313.381/s)
ok 1 - POST /word/foo
ok 2 - 200 OK
[2021-02-28 18:13:50.83531] [79162] [debug] [G0qyFA0G] GET "/word/foo"
[2021-02-28 18:13:50.83568] [79162] [debug] [G0qyFA0G] Routing to controller "Local::Dictionary::Microservice::Controller::Word" and action "define"
[2021-02-28 18:13:50.85745] [79162] [debug] [G0qyFA0G] 200 OK (0.022096s, 45.257/s)
ok 3 - GET /word/foo
ok 4 - 200 OK
ok 5 - exact match for content
[2021-02-28 18:13:50.86182] [79162] [debug] [MoedWcOi] DELETE "/word/foo"
[2021-02-28 18:13:50.86224] [79162] [debug] [MoedWcOi] Routing to controller "Local::Dictionary::Microservice::Controller::Word" and action "remove"
[2021-02-28 18:13:50.86337] [79162] [debug] [MoedWcOi] 200 OK (0.001535s, 651.466/s)
ok 6 - DELETE /word/foo
ok 7 - 200 OK
[2021-02-28 18:13:50.86662] [79162] [debug] [G2lH8gyw] GET "/word/foo"
[2021-02-28 18:13:50.86684] [79162] [debug] [G2lH8gyw] Routing to controller "Local::Dictionary::Microservice::Controller::Word" and action "define"
[2021-02-28 18:13:50.86964] [79162] [debug] [G2lH8gyw] 404 Not Found (0.002989s, 334.560/s)
ok 8 - GET /word/foo
ok 9 - 404 Not Found
1..9
ok
All tests successful.
Files=1, Tests=9,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.46 cusr  0.10 csys =  0.59 CPU)
Result: PASS

Congratulations! You now have a simple microservice for storing and retrieving definitions of words. You can now write a web front-​end in HTML and JavaScript, or perhaps another microservice that consumes this one.

(As an exercise, write the /health route defined in our OpenAPI document. It calls a heartbeat method in a controller class named Monitoring.)

What next?

  • You may not want to store your definitions in a DB file in your project; consider making it a configurable option. Or use one of the many modules on CPAN to choose a completely different backend.
  • Add methods to list some or all definitions, using hypertext as the engine of application state (HATEOAS) to render links where appropriate.
  • Support multiple definitions of the same word, or expand the API to respond with synonyms and antonyms. (Just make sure to increment the version in the OpenAPI document and add some way for clients to specify the version to indicate you’re making a breaking change!)

Did you have any trouble following along? Got stuck on the installation steps or somewhere else? Please leave a comment below and I’ll try to help.

top view of assorted gadgets on desk

A microservice can be thought of as the distributed computing implementation of the Unix philosophy of Do one thing and do it well.” Usually modeled around specific business domains, a well-​designed set of microservices each have their own lifecycle and communicate with each other and with consuming applications via technology-​agnostic protocols like HTTP.

Because of this, a microservice may be implemented using whatever is the best programming language, database, or another environment to fit the job. Perl can be an excellent choice for development because of its strengths as a battle-​tested, expressive multi-​paradigm language with a broad open-​source library of modules.

In this series of articles, we’ll be using Perl, the Mojolicious web framework, and the OpenAPI (formerly Swagger) specification for describing web services to develop an example microservice that adheres to the REST architectural style for communication over HTTP. Some knowledge of Perl and web development is assumed, and we’ll be linking to supporting documentation and literature as we go.

In a full microservices implementation, we would start by modeling the business domains being serviced and the explicit boundaries between them. For the purposes of this series, we are limiting ourselves to developing only one microservice with a trivial purpose—a toy, if you will. This service will act as a simple dictionary allowing consumers to both query for the definitions of words as well as add new entries.

We could envision a variety of consumers: a web front-​end, a mobile app, maybe even SMS text messaging. The key is that the design of our application programming interface (API) should not dictate the implementation or lifecycle of any of these consumers, and it should be easy for their developers to understand its specification.

To that end, we turn to OpenAPI, which gives both our microservice and its consumers an unambiguous machine-​readable description of the interface without the need for additional code or documentation. And as we’ll see later, we’ll use that very same specification to drive our web framework to produce HTTP routes that accept valid input and produce valid output.

A full OpenAPI document can be written in either JSON or YAML format, but it must ultimately be able to be represented in both formats, so there are some limitations described in the spec. Here’s the OpenAPI document for our dictionary microservice in YAML:

openapi: 3.0.3
info:
  title: Dictionary
  description: The PhoenixTrap.com dictionary microservice
  version: 1.0.0
  license:
    name: Artistic License 2.0
    url: https://www.perlfoundation.org/artistic-license-20.html
paths:
  /health:
    get:
      summary: Check if this service is online
      x-mojo-to: monitoring#heartbeat
      responses:
        200:
          description: All systems operational
          content:
            application/json:
              schema:
                type: object
        500:
          description: Something is wrong
  /word/{word}:
    get:
      summary: Get the definition of a word
      x-mojo-to: word#define
      parameters:
      - $ref: '#/components/parameters/word'
      responses:
        200:
          description: Found word
          content:
            application/json:
              schema:
                type: string
        404:
          description: Could not find word
    post:
      summary: Add or replace the definition of a word
      x-mojo-to: word#save
      parameters:
      - $ref: '#/components/parameters/word'
      requestBody:
        description: Definition of a word
        required: true
        content:
          application/x-www-form-urlencoded:
            schema:
              type: object
              properties:
                definition:
                  type: string
      responses:
        200:
          description: Word saved
    delete:
      summary: Delete an entry from the dictionary
      x-mojo-to: word#remove
      parameters:
      - $ref: '#/components/parameters/word'
      responses:
        200:
          description: Word deleted
components:
  parameters:
    word:
      description: A word in the dictionary
      in: path
      name: word
      required: true
      schema:
        type: string

(I worked on this in the free online Swagger Editor, which checks your syntax, previews what API documentation might look like, and offers suggestions if it finds errors in your document.)

Most of the above should be fairly self-​explanatory to any web developer, and you can check the OpenAPI specification for required and optional fields and their allowed values. The only odd field listed is x-mojo-to; we’ll be using the Mojolicious::Plugin::OpenAPI module to read that field when generating routes to controllers in the Mojolicious web framework.

In the next installment, we’ll actually be writing some Perl by building our Mojo application and tests.