Category Archives: Node.js

Documentation Driven Development

Documentation Driven Development

At adopting a new and growing trend, “documentation driven development” via “swagger” and now the newly updated specification, “OpenAPI 3”.

Documentation driven development (DDD) is not something new (some of its earliest concepts date back to 2011), but up until recently it never had an officially accepted name, nor an industry standard for the world to agree upon. It is the process of describing an API’s endpoints and what it expects as a request and should respond with. This enables a front-end team to easily use an API within their apps and mock the response.

Documentation driven development boils down to a two high level and basic principles:

  • Design the API first, then build it
    • The API design must adhere to the OpenAPI Specification (formerly known as Swagger)
    • The yaml file can be broken up with swagger-chunk
    • The finished result of the API design should be a Yaml or JSON file
    • The actual API routes should be built on what is documented in the design
    • The client, eg the browser, should communicate with the API via a generated file, built from a tool such as swagger-code-gen
  • An endpoint not documented should not exist
    • A JavaScript front-end engineer using a generated DDD API access file should not look outside of this file to access the API
    • As all routes must be documented, any other routes can be seen as not required &/or dangerous

Adhering to DDD simultaneously increases development speed whilst solving a growing issue regarding how the front-end and back-end teams both build and communicate. Further more this is not limited to company cross-team dynamics but also expandable to cross-company communications.

Looking to the future

DDD has given us some very big gains as described, but what could the future look like and how do we aim to streamline this process yet further.

Currently the API design must be written in a single Yaml file. When the API gets bigger than trivial the need to break the file down into sizable chunks becomes a must, this is where swagger-chunk comes in. swagger-chunk allows the developer to write many smaller and managable Yaml files which swagger-chunk then combines together into one.

Swagger-chunk is one step in the right direction, but it can be easy to get lost in a myriad of similar looking files. Further more, designing an API does not nesasarily depend on skill of a developer if there were an alternative to writing pure Yaml chunks…. Enter openapi-gui.

Openapi-gui is a graphical user interface to building the API design. The API designer does not require anything more than a web browser and technical knowledge on what a RESTfull API should be and deliver. In laymans terms, the API design can now be realised by a skilled technical product owner

 

Server side rendering with phpv8/v8js on Ubuntu

https://vuejsdevelopers.com/2017/11/06/vue-js-laravel-server-side-rendering/ requires phpv8/v8js. Here is how to install the latest (at the time of writing) with php7.1.

Add the latest ppa from this guy: https://launchpad.net/~pinepain

Update the sources:

Install libv8:

Install and compile v8js

Optionally remove the files from the install from /tmp.

After this the SSR of the example vuejs app should work.

__dirname resolving to root of disk… /

Building an electron app you will find most of the examples refer to loading with something like:

The problem is, if you build with webpack.. then __dirname will actually resolve to the root of the disk.

To resolve this there is a simple hack/fix:

In your webpack config, add these two flags as false and things will start working.

Electron packager tutorial

Example electron app

This guy made a nice tutorial on electron with the code here, a perfect starting point if you’ve just arrived at electron

https://github.com/bradtraversy/electronshoppinglist

Install the dependencies to your system and app:

Adding the scripts to your package.json

And run

 

To create a package for debain, aka .deb

This guys has written a nice blog post on it:

https://www.christianengvall.se/electron-installer-debian-package/

 

Multipart form breaks nodejs express expected format

Multi-part forms usually break express as the expected body is not present.. This is also true for a JS app posting to Node if the JS app builds a FormData object:

https://www.npmjs.com/package/express-form-data

The above npm package fixes this. It acts as a middleware to auto format incoming requests into the nice body and file format you are expecting. In your press middleware loader add the following:

 

List node app scripts cli without opening the package.json

This was annoying me so much I was about to reach for my IDE and write my own global package to just list the scripts.. good job i reached for google first!

So many times you pick up a new project, or one lands on your lap and the usual thing happens.. no std script names. The usual trick:

You could pipe this to grep to pull out all you wanted but lifes too short for that.

Enter the likes of: https://www.npmjs.com/package/npm-ls-scripts. It is quite old code and code be optimized but it does a job and now just type:

Ahhh… itch has now been itched 🙂

https://www.npmjs.com/package/npm-ls-scripts

Testing via Mocha and Chai in ES5 and ES6

Testing via Mocha and Chai in ES5 and ES6

Testing is a crucial part for developing, for JavaScript as well. Since the foundation theories of testing is almost the same regardless of the languages you are using. I will focus on introduction to Mocha and Chai, and environment setting up in this article. And for both ES5 and ES6. And the library we are using is Mocha and Chai. Mocha is a BDD testing framework and Chai is an assertion library. And at the end, I will talk about why I don’t use Jasmine for testing.

 

1. About the versions:

  • “node.js”: “4.6.2”,
  • “babel-preset-latest”: “6.16.0”,
  • “babel-register”: “6.18.0”,
  • “chai”: “3.5.0”,
  • “mocha”: “3.2.0”

2. One minute for Mocha

There are so many little piece in Mocha, but you should know the following 3 for your first test. And conquer the rest at the homepage of Mocha, not that long.

2.1 Make your context via describe()

Your testing suites may consist of many parts. And you could distinguish them in a clear manner via describe(), you needs 2 parameters for it, the first one is the description and the second one is a function which will contain your tests afterwards.

1
2
3
4
5
6
7

describe(“Start to test function A”, function(){
// Your tests for fnA starts here
})
describe(“Start to test function B”, function(){
// Your tests for fnB starts here
})

If you have more cases to address, you can use another function called context() to do the trick:

1
2
3
4
5
6
7
8

describe(“Start to test function A”, function(){
context(“Test case 1”, function(){
// Your tests for case 1
})
context(“Test case 2”, function(){
// Your tests for case 2
})
})

2.2 Write your tests in it()

Your real code for tests locates in it(), it has a same function signature as describe().

1
2
3
4
5

describe(“Start to test function A”, function(){
it(“should return 2 when we pass 1 and 1”, function(){
expect(addTwo(1, 1)).to.be.equal(2)
})
})

Furthermore, if you are writing an asynchronous testing. Which means your testing code may ends later, but the whole block won’t wait your tests to finish. You can pass a done to the callback function of it() to fix it.

1
2
3
4
5
6

describe(“Start to test function A”, function(){
it(“should return 2 when we pass 1 and 1”, function(done){
expect(addTwo(1, 1)).to.be.equal(2)
done()
})
})

2.3 You can hook your tests to do some preparation

With its default “BDD”-style interface, Mocha provides the hooks before(), after(), beforeEach(), and afterEach(). These should be used to set up preconditions and clean up after your tests.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

// Codes below are from Mochajs.org
describe(‘hooks’, function() {
before(function() {
// runs before all tests in this block
})
after(function() {
// runs after all tests in this block
})
beforeEach(function() {
// runs before each test in this block
})
afterEach(function() {
// runs after each test in this block
})
// test cases
it(“should return something”, function(){
// test codes here
})
})

3. One minute to Chai

Chai focus on assertion, it provides with 3 types of assertions. Codes below are from http://Chaijs.com/guide/styles

3.1 Assert

This one has a similar feeling as the build-in assert in node.js.

1
2
3
4
5
6
7
8
9

var assert = require(‘chai’).assert
, foo = ‘bar’
, beverages = { tea: [ ‘chai’, ‘matcha’, ‘oolong’ ] };
assert.typeOf(foo, ‘string’); // without optional message
assert.typeOf(foo, ‘string’, ‘foo is a string’); // with optional message
assert.equal(foo, ‘bar’, ‘foo equal `bar`’);
assert.lengthOf(foo, 3, ‘foo`s value has a length of 3’);
assert.lengthOf(beverages.tea, 3, ‘beverages has 3 types of tea’);

3.2 Expect

The BDD style assertion of expect() enable you to represent your tests in a more human-reading-friendly way.

1
2
3
4
5
6
7
8

var expect = require(‘chai’).expect
, foo = ‘bar’
, beverages = { tea: [ ‘chai’, ‘matcha’, ‘oolong’ ] };
expect(foo).not.to.be.a(‘number’);
expect(foo).to.equal(‘bar’);
expect(foo).to.have.length(3);
expect(beverages).to.have.property(‘tea’).with.length(3);

3.3 Should

Provides another approach to address your tests description. May have some issues when used with Internet Explorer, so be aware of browser compatibility.

1
2
3
4
5
6
7
8

var should = require(‘chai’).should() //actually call the function
, foo = ‘bar’
, beverages = { tea: [ ‘chai’, ‘matcha’, ‘oolong’ ] };
foo.should.not.be.a(‘number’);
foo.should.equal(‘bar’);
foo.should.have.length(3);
beverages.should.have.property(‘tea’).with.length(3);

3.4 Languages chains

You can chain your description via the following methods. Here is a little disadvantage of using Chai. Non-native speaker may have a trouble on combine these words to a sentence, but seems not a big deal, they are very basic English.

  • to
  • be
  • been
  • is
  • that
  • which
  • and
  • has
  • have
  • with
  • at
  • of
  • same

4. Set up for ES5 Testing

Very easy to set up.

4.1 First, you install them

1
2
3
4
5
6
7

# use NPM
npm init –yes
npm install mocha chai –save-dev
# or use fancy Yarn
yarn init –yes
yarn add mocha chai –dev

4.2 Second, import them in your tests and write a simple test

Create a file named test.js locates in test/ in your project folder.

1
2
3
4
5
6
7
8
9
10
11
12

var expect = require(“chai”).expect
var should = require(“chai”).should()
var addTwo = require(“../index”).addTwo
describe(“Test the behavior of addTwo()”, function () {
it(‘should return 2 when given 1 and 1 via expect()’, function () {
expect(addTwo(1, 1)).to.be.equal(2)
})
it(‘should not return 3 when given 1 and 1 via should()’, function () {
addTwo(1, 1).should.not.be.equal(3)
})
})

4.3 Third, write your function which needs to test.

The above tests will fail since there is no such addTwo(). It doesn’t matter, we will add one now.
Create a file named index.js locates in the root of your project folder with the following codes:

1
2
3

export addTwo = function (num1, num2) {
return num1 + num2;
}

4.4 Fourth, add the following section to your package.json.

If you already have a scripts section, you can replace it with the following one.

1
2
3

“scripts”: {
“test”: “./node_modules/mocha/bin/mocha test/*.js || exit 0”
},

4.5 Run your tests via command line

1
2
3
4

npm run test
# Or, just:
npm test

You can do the second trick since the test is a reserved keyword of NPM.

5. See the fancy result

1
2
3
4
5
6
7
8

> testMocha@1.0.0 test /Users/albertgao/codes/node/testMocha
> ./node_modules/mocha/bin/mocha test/*.js || exit 0
Test the behavior of addTwo()
✓ should return 2 when given 1 and 1 via expect()
✓ should not return 3 when given 1 and 1 via should()
2 passing (8ms)

6. Set up for ES6 Testing

Testing for ES6 takes few more steps since you need to transpile your ES6 code to ES5.

6.1 First, you install them

1
2
3
4
5
6
7

# use NPM
npm init –yes
npm install mocha chai –save-dev
# or use fancy Yarn
yarn init –yes
yarn add mocha chai –dev

6.2 Second, import them in your tests and write a simple test

Create a file named test.js locates in test/ in your project folder.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

import chai from “chai”
import {addTwo} from “../index”
let expect = chai.expect
let should = chai.should()
describe(“Test the behavior of addTwo()”, function () {
it(‘should return 2 when given 1 and 1 via expect()’, function () {
expect(addTwo(1, 1)).to.be.equal(2)
})
it(‘should not return 3 when given 1 and 1 via should()’, function () {
addTwo(1, 1).should.not.be.equal(3)
})
})

Something interesting happens here, I imported chai, then apply its two functions to local variables. Why not just import {expect,should} from "chai", saddly, you can do it for expect but not should,

According to the official docs now:

It isn’t possible to chain a function call from an ES2015 import statement – it has to go on its own line.

But I saw a request on Github, which will enable import "chai/should" in the future, hopefully in the 4.0 version. It mentioned in official docs. But I tried with no luck.

Second, I didn’t use arrow functions here since the nature of the arrow function, you will lose the context binding, let’s say you want to use the built-in this.timeout(200) to structure your tests. Then you shouldn’t use the arrow function even you are written in ES6. But if not, feel free to use it.

6.3 Third, write your function which needs to test.

The above tests will fail since there is no such addTwo(). It doesn’t matter, we will add one now.
Create a file named index.js locates in the root of your project folder with the following codes:

1
2
3
4
5

let addTwo = (num1, num2) => {
return num1 + num2;
}
export {addTwo}

6.4 Fourth, add the Babel support

1
2
3
4
5

# use npm
npm install babel-preset-latest babel-register –save-dev
# or use fancy Yarn, -D === –dev
yarn add babel-preset-latest babel-register -D

6.5 Fifth, create a .babelrc file at the root of your project folder

With the following contents:

1
2
3

{
“presets”: [“latest”]
}

6.6 Sixth, add the following section to your package.json.

If you already have a scripts section, you can replace it with the following one.

1
2
3

“scripts”: {
“test”: “./node_modules/mocha/bin/mocha test/*.js –require babel-register –reporter spec || exit 0”
},

The idea here is easy, Mocha just uses babel-register to transpile the file on the fly. Via this approach, you can write you tests and codes both in ES6, and tests them without pre-transpiling the code.

6.7 Run your tests via command line

1
2
3
4

npm run test
# Or, just:
npm test

You can do the second trick since the test is a reserved keyword of NPM.

7. See the fancy result

1
2
3
4
5
6
7
8

> testMocha@1.0.0 test /Users/albertgao/codes/node/testMocha
> ./node_modules/mocha/bin/mocha test/*.js –require babel-register –reporter spec || exit 0
Test the behavior of addTwo()
✓ should return 2 when given 1 and 1 via expect()
✓ should not return 3 when given 1 and 1 via should()
2 passing (8ms)

8. Why not Jasmine?

A little off-topic talking before ending. Why not Jasmine? It is good, it is full featured. Has built-in assert, mock(spy), ajax call, etc. And it’s easy to configure too. Sometimes just as same as Mocha, but with Mocha, you can choose your own favourite assertion library, like Chai or better-assert. And choose mock library like Sinon.js, and even for the reporter part, you can use different reporter for a different outlook, even for exporting a HTML report.

So the main differences between Mocha and Jasmine is that you don’t this so called javascript fatigue anymore with Jasmine. But this is just why we love JavaScript, right? (Not wierd, think about it. 🙂

Express validation + Joi for node applications

If you are coming from Laravel to Node you will likely be hunting for a few packages that enable you validate incoming requests and ensure standardised out for your API.

First up, validating your incoming requests with “express-validation” and “joi”.

Together these things can be used with your expressJs app to validate incoming requests and clearly describe to new developers on your project what the expected input to a rout should be:

Here is a full article on the topic, es5 and es6

Second standardizing your api output

If like me you prefer to stick to industry standards unless required, then you are probably in favor  on JSON API

There are lots and lots on implementations of the JSON API standard, but one in particular for me stands out for NodeJS: https://www.npmjs.com/package/jsonapi-serializer

This is a popular node package and most importantly to me, it is decoupled from any ORM/ODM. In other words you can get familiar one implementation and use it for almost any NodeJS app.

 

webpack, es6, babel, sass and custom webfont

Webpack has been gaining a heck of a lot of popularity over the past year. Webpack is a different breed to the messy likes of “gulp” or “grunt”, it prevents developers from creating long and whacky build files. One other option that also prevents whacky build files is quilk. 

Webpack can be hard to get your head into.. its like taking the std build tools but looking at them from reverse. Getting the basics up and running is fairly novice, but getting SASS with Webfonts can be a little more tricky.

In this git-repo I have put together a demo, es6+sass+custom webfonts.

https://github.com/jdcrecur/demo-webpack

Enjoy 😉

 

Webpack – SASS – Bootstrap-sass

In a previous post I put together a sass compiling webpack file… http://www.beingyesterday.com/webpack-and-sass/

But you will run into more fun and games if you want to use bootstrap-sass with this. The fun being more like a headache actually.

Anyway… if you add bootstrap-sass as an include in any of your .scss files the result will be that webpack cannot load/process any of the fonts. To get around this we need a new webpack loader and a new dev dependency in the package.json:

The extra loader above creates the rel. files in a /font directory.

 

The additional loader also requires the ‘url-loader’ adding to the package.json