QAVideos (3) – Extending the Data Model and Using Open API Initiative (OAI) in Node.js

2 Replies

(updated: 20170319) WIP
This is part 3 in a series to build a sample application called QAVideos using API Connect. QAVideos is a Question and Answer application that uses videos instead of text. Think of QAVideos as StackOverflow meets Youtube. QAVideos will use LoopBack, API Connect, Open API Initiative (OAI formerly Swagger.io), OpenWhisk, Object Storage.

  • In part 1 ‘QAVideos (Part 1), Adding User Management to Node.js’, I added User Management to a Node.js app using API Connect.
  • In part 2 ‘QAVideos (Part 2), Adding a Custom Model and ORM in Node.js‘, I added a custom data model, i.e. a Video model, and used ORM to persist the model to a PostgreSQL database.
  • In part 3 ‘QAVideos (Part 3), Extend Model and Using Open API Initiative (OAI) in Node.js‘, I extend the built-in User model with a Member, add a Question and Answer based on the Video model, and add a Sticker, while using the Open API Specification to define and manage the application and APIs.
  • In next parts, the idea is to add other possibly features: deploy the OAI definition file to Bluemix via API Connect, add Object Storage for Video support, create an Ionic/Apache Cordova based mobile client, add an automated build, deployment and test script, add a Content Delivery Network (CDN) to the backend, add event-driven OpenWhisk APIs, containerize the application, add chat, and possibly more.

Prerequisites

  • Node, npm, and API Connect must be installed,
  • Check if the ‘apic’ tool is installed, by running ‘apic -v’ from the commandline. If not installed, follow the installation instructions, here.
  • Get the source code from part 2 of this tutorial.
  • Install the ‘npm’ dependencies,
    npm install
  • Configure the file ‘~/server/datasources.json’,
  • Test if QAVideos (part 2) is running correctly by typing ‘apic start’ in the root directory, open a browser and go to ‘http://0.0.0.0:4001’ in your browser.

Table of Content

  1. The OAI Definition File
  2. Create the Extended Data Model

Open API Initiative (OAI, based on Swagger)

The OAI is based on Swagger.io, an API Framework. The OAI is an open API definition standard. Frameworks like API Connect use the OAI definition file to generate the server, client, data model and documentation.

API Connect

LoopBack is an extensible, open-source Node.js framework. LoopBack is included in the API Connect framework and can be accessed by running the ‘apic loopback’ command. Many of the former LoopBack cli command ‘slc’ are now supported directly via the API Connect cli ‘apic’ command or via the ‘apic loopback’ command.

Adding Models

The class diagram for QAVideos is as follows.

I want to extend the previous data model and add the following new models:

  • Member, an extension of the built-in User model,
  • Question and Answer, both an extension of the Video model, and
  • Sticker, a new model to capture user reactions like emojis and stickers.

1. The OAI Definition File

If you have followed part 1 and part 2 of the QAVideos tutorial, or if you downloaded the source code of part 2, the apic cli tool of API Connect has generated among other a directory ‘~/.apiconnect’ with a ‘config’ file. This ‘config’ file can be used to override the installation config file in the user’s root directory, on a Mac ‘/users/<username>/.apiconnect’.
API Connect also generated a ‘~/definitions’ directory with 2 yaml files: ‘<projectname>-product.yaml’ and ‘<projectname>.yaml’. YAML is a human-readable configuration format that is widely used for configuration file. Let’s take a closer look at the OAI files, cause this is the central spine of the QAVideos application and will let us deploy and manage the application in the future.

In API Connect jargon, the Product is the published version of the Draft. You create and edit the Draft version of the API, when you are finished editing, you publish the Draft of the API to create the Product. The Product contains the metadata for the current Product.

In your IDE, open the ‘<projectname>-product.yaml’ file.

  • ‘apis’, you see that the product yaml refers to the APIs yaml in the ‘apis’ property,
  • ‘visibility’, visibility is used to define the authentication level and the visibility of your APIs,
  • ‘plans’, the product yaml also includes a ‘plans’ property for publishing and managing your APIs via the API Connect service on Bluemix.

In your IDE, open the ‘<projectname>.yaml’ file. The definition of the properties in the current version of the OAI definition file, version 2.0.0, can be found in the OAI documentation.

Looking at the highest level properties:

  • ‘paths,’ the ‘paths’ section is required, there must be at least 1 endpoint.
  • ‘definitions,’ the definitions section holds the definitions of references. A reference to the Video model for instance can be create with ‘$ref: ‘#/definitions/Video’,
  • ‘basepath,’ the ‘basePath’ property ‘/api’ is used by API Connect to prepend the API endpoints for those endpoints defined in the ‘paths’ section,
  • ‘swagger,’ this property defines the version of Swagger that the OAI definition file complies to,
  • ‘schemes’ specifies the transfer protocol of the API like ‘http’ or ‘https’,
  • ‘consumes’ and ‘produces’ define the media types or MIME types that the API accepts and returns, in this case API Connect defined the default ‘application/json,’ other MIME types can be found at the IANA Mime Types site,
  • ‘security’ defines the security scheme that is required to access the API, a list of key-value pairs, that comply to the security scheme, for instance ‘oauth2’.

Looking at the ‘paths’ section:

  • Under the ‘paths’ you obviously see the paths or endpoints of the different resources. Some of the paths include parameters notated between curly brackets ‘/Users/{id}’,
  • Each path has one or more HTTP methods defined complying to the REST or HTTP definition for CRUD operations. To define to create a User you specify under the ‘/Users’ path a ‘post:’ method,
  • Each method defines several properties:
    • ‘tags,’ is used to categorize this path and method in the documentation that is generated from the OAI definition file,
    • ‘summary,’ is free text to explain the purpose of the method,
    • ‘operationId,’ is a unique string to identify the operation,
    • ‘parameters,’ define the URL or body parameters,
    • ‘responses,’ define the response object per HTTP response code, ‘200’ specifies a successful response, ‘404’ not found, etc. HTTP response codes can be found at the w3.org’s HTTP specification site,
    • ‘deprecated,’ default value is false.

Looking at the parameters:

  • ‘name,’ is the name of the parameter, which is also used to reference the parameter, see the ‘in’ property,
  • ‘in,’ refers to the location where the parameter is used, possible values are “query”, “header”, “path”, “formData” or “body”,
  • ‘description,’
  • ‘required,’
  • ‘type,’ value must be “string”, “number”, “integer”, “boolean”, “array” or “file”,
  • ‘format,’ an extended format of ‘type’, see the list of formats,
  • ‘schema,’ instead of defining the model’s properties you can define a schema and refer to a definition.

Looking at the data models in the definitions sections:

  • ‘properties,’ is a list of data model properties, which are defined by the property name with the type and format,
  • ‘required,’ defines an array of the properties that are required,

2. Create the Extended Data Model

Now you understand the generated OAI definition file, we can add the complete model. I need to define 5 models: Member, Video, Question, Answer and Sticker.

Open the command-line and change to the project directory. Create the Member model.
apic create --type model Member
Make sure you select the ‘postgresdb’ as the data-source to attach Member to, select User as the model’s base class, and add a property userLevel of type string that is not required.

Open your IDE and open the file ‘~/common/models/member.json’.
{
"name": "Member",
"base": "User",
You see that Member is now a child of User. Open the file ‘~/server/model-config.json’ and change the visibility of User to ‘public: false’, because the built-in User is only the base model, but the Member is the model we want to expose.
"User": {
"dataSource": "postgresdb",
"public": false
},
Note that any changes to the model must be migrated to the datasource still. So when we are done making all changes to the schema, I will update the schema in the postgresql database using the automigrate tool.

Next, create the Question model.
apic create --type model Question

Make sure you select the ‘postgresdb’ as the data-source to attach Question to, and select Video as the model’s base class.

Next, create the Answer model.
apic create --type model Answer

Make sure you select the ‘postgresdb’ as the data-source to attach Answer to, and select Video as the model’s base class. Do not create a property questionId, cause this is a relation property that I will create when I add the relations.

Next, create the Sticker model.
apic create --type model Sticker

Make sure you select the ‘postgresdb’ as the data-source to attach Sticker to, select PersistedModel as the base class, add a property stickerType of type string that is required amd which default value is ’emoji-happy’. Do not create the properties memberId and questionId, cause these are relation properties that I will create when I add the relations.

3. Add Relations to the Model

I need to add the following relations:

  • Member has many Question,
  • Member has many Answer,
  • Question has many Answer,
  • Video has many Sticker,
  • Sticker has one Member, and lastly,
  • remove the belongs to relation from Video to User, cause this has been replaced by the Member has many Question, and Member has many Answer relations.

Using the ‘apic loopback:relation’ command, add a ‘has many’ relation from Member to Question.

Add a ‘has many’ relation from Member to Answer.

Add a ‘has many’ relation from Question to Answer.

Add a ‘has many’ relation from Video to Sticker.

Add a ‘has one’ relation from Sticker to Member.

Open your IDE and open the ‘~/common/models/video.js’ file. Remove the ‘videoBelongsToUser’ relation.
"videoBelongsToUser": {
"type": "belongsTo",
"model": "User",
"foreignKey": "videotouserid"
},
So that the relations property in the ‘video.json’ file looks now as follows:
"relations": {
"videoHasStickers": {
"type": "hasMany",
"model": "Sticker",
"foreignKey": "videoHasStickersFk"
}
},

Set Visibility

Because Question and Answer are Generalizations of Video, and Member is a Generalization of User, we want to hide Video and User from public visibility. Open your IDE and edit the ‘~/server/model-config.json’ file and add the property ‘public: false’ to User and Video.
"Video": {
"dataSource": "postgresdb",
"public": false
},
"User": {
"dataSource": "postgresdb",
"public": false
},

Adding ACL

The Member, Question, Answer, and Sticker still need an ACL. Open the ‘~/common/models/video.js’ file and copy the ‘acls’ property.

then in the directory ‘~/common/models/’ open respectively the files ‘sticker.json’, ‘member.json’, ‘question.json’ and ‘answer.json’ and replace the empty ‘acls’ property with the ‘acls’ values above.

Autoupdate Model to PostgreSQL Schema

Edit the file ‘~/server/bin/automigrate.js’.

Run the automigrate tool from the commandline:
node server/bin/automigrate.js
After running the automigration tool, the schema was dropped and created from the models in the automigrate tool. The ‘Member’ model has replaced the default User model, and the Answer and Question models have replaced the Video model.

If you do not want to re-create the schema from scratch, instead of ‘automigrate’ you can use the ‘autoupdate’ tool.
var app = require('../server');
var dataSource = app.dataSources.postgresdb;
dataSource.autoupdate([
'Sticker'
], function(err) {
if (err) throw err;
});

Regenerate the Angular Services

With new models added to QAVideos, from the command-line re-run the ‘lb-ng’ command to add an Angular services SDK for the models in the QAVideos app.
$ lb-ng server/server.js client/js/lb-ng-services.js
Loading LoopBack app "/Users/remko/dev/src/github/qavideos/server/server.js"
Generating "lbServices" for the API endpoint "/api"
Warning: scope Member.accessTokens targets class "AccessToken", which is not exposed
via remoting. The Angular code for this scope won't be generated.
Saving the generated services source to "/Users/remko/dev/src/github/qavideos/client/js/lb-ng-services.js"

Update Angular Controllers and Factories

To test the changes, from the command-line run the ‘apic start’ command and in a browser open the URL ‘http://0.0.0.0:4001‘.

Leave a Reply

Your email address will not be published. Required fields are marked *