Redundancies & Update ripple effects

in node package manager

Amir Saboury

npm is the package manager for javascript, and is default for Node.js

near 111,000 package.

javascript: not just for browsers any more!

 

Semantic Versioning

Increment the:
MAJOR version when you make incompatible API changes.
MINOR version when you add functionality in a backwards-compatible manner.
​PATCH version when you make backwards-compatible bug fixes.

MAJOR.MINOR.PATCH

Package.json

​"dependencies": {
    "primus": "*",
    "async": "~0.8.0", // >= 0.8.0 < 0.9.0
    "express": "4.2.x", // >=4.2.0 < 5.0.0
    "vows": "^0.7.0", // >=0.7.0 < 1.0.0
    "assume": "<1.0.0 || >=2.3.1 <2.4.5 || >=2.5.2 <3.0.0",
    "winston": "git://github.com/flatiron/winston#master",
    "bigpipe": "bigpipe/pagelet",
    "plates": "https://github.com/flatiron/plates/tarball/master"
}

Dependency Hell!

A@1.0.0
|
+-- B@1.0.0
|   |
|   `-- D@1.0.0 \
|                \
`-- C@1.0.0       > Conflict
    |            /
    `-- D@2.0.0 /

RQ1

What is the redundancy percentage in each module?

A@1.0.0
|
+-- B@1.0.0
|   |
|   `-- D@1.0.0 \
|                \
`-- C@1.0.0       > Redundancy
    |            /
    `-- D@1.0.0 /

RQ2

How long it takes for each module to update their dependencies after an update in one of the dependencies?

Maximum and Average time

a module gets updated after dependencies are updated

Model

Got the graph

by Breadth First Search

 

Used 10 most starred modules, 10 most depended upon modules and 10 random modules as roots.

For each module

Express

}

dependet on

{

dependet to

For the latest version

For all versions

Covering

78%

of all the modules in the registry

  • Dead Modules
  • Test Modules
  • Non-JS Modules

The rest could be:

For each module

timeline

v0.0.1
21aug
v0.0.2
29aug
v0.1.2
12dec

RQ1

Express

1.2.2

1.2.3

1.2.4

Connect @ v.0.5.3

Connect @ v.0.5.3

Connect @ v.0.6.1

RQ1

Using a Depth-First-Search

and flattening the result in each node

"mime": {
  "1.2.11": 2
},
"connect": {
  "0.5.3": 2,
  "~0.6.1": 1
}

RQ1

2,879,246

The total number of dependency relations

2,404,194

If we do not consider different versions

 

40,505,974

If we count duplicates twice

102

RQ1

For every module

[number of duplicates - number of dependencies]

52%

zero redundancy

75%

less than 15 redundant modules

Max: 473987

RQ2

B

v0.1.1
v0.1.2
v0.1.3

A

v0.0.1
v0.0.2
v0.1.0
v0.1.4
v0.2.1
v0.2.3
v0.2.5

A depends on B

v0.0.3

RQ2

71.6days

The overal average time of propagation time

82.4days

The average time of "maximum update propagation time"

for each module's dependency

RQ2

50% below 21.6days

25% below 2.2days

The average time of average propagation time for each module's dependency

RQ2

50% below 26.4days

25% below 2.9days

The average time of maximum propagation time for each module's dependency

RQ2

50% below 37.4days

25% below 3.7days

The maximum time of maximum propagation time for each module's dependency

52% zero redundancy

75% less than 15 redundant modules

50% below 21.6days
25% below 2.2days
The average time of average propagation time for each module's dependency

50% below 26.4days
25% below 2.9days
The average time of maximum propagation time for each module's dependency

50% below 37.4days
25% below 3.7days
The maximum time of maximum propagation time for each module's dependency

Copy of npm

By amir

Copy of npm

  • 960