那些架設Composer鏡像小事

PHP也有Day#58

2020/12/16

Peter

Some Small Stuffs About Building Composer Mirror

Slide

Outline

  • About me
  • What's Composer?
    • Composer Installation
    • Composer Ecosystem
  • How Composer Operating?
  • What's Composer Mirror?
  • Building a Composer Mirror-part1
  • Building a Composer Mirror-part2
  • Composer V1 Issues
  • Composer V2 Benefits
  • Selecting Hosting Providers Best Practices

About me

  • Peter
  • GitHub
  • Active open source contributor
  • LaravelConf Taiwan member
  • An associate engineer
    • DevOps
    • Back-end
    • System Architecture Researching
    • Web Application Security
    • PHP, Python and JavaScript
  • Smart Grid Technology (2017~)

What's Composer?

Composer Installation

curl -sS https://getcomposer.org/installer | sudo php \
	-- --install-dir=/usr/local/bin --filename=composer
curl -sS https://getcomposer.org/installer | php -- --1

Manual Composer Installation

wget https://getcomposer.org/download/2.0.8/composer.phar

Composer Ecosystem

Composer Ecosystem

How Composer Operating?

How Composer Operating?

How Composer Operating?

How Composer Operating?

How Composer Operating?

Install laravel/laravel package again

How Composer Operating?

How Composer Operating?

php ./composer.phar require laravel/laravel -vvv

How Composer Operating?

php ./composer.phar require laravel/laravel -vvv

How Composer Operating?

php ./composer.phar require laravel/laravel -vvv

How Composer Operating?

php ./composer.phar require laravel/laravel -vvv

How Composer Operating?

php ./composer.phar require laravel/laravel -vvv

How Composer Operating?

rm -rf vendor/ composer.lock

How Composer Operating?

php composer.phar clearcache

php ./composer.phar require laravel/laravel -vvv

Brief summary

How Composer Operating?

  • Reading required JSON setting files

  • Downloading provider-%hash% files

  • Reading composer.json file

  • Dependency and versioning checking

  • Installing required package

    • Downloading or Reading package zip archive files

What's Composer Mirror?

Motivation

Back to 2020/06/22

composer issue#9003

Packagist Tweet

composer issue#9003

Packagist Mirrors

{
  "packages": [],
  "notify": "https://packagist.org/downloads/%package%",
  "notify-batch": "https://packagist.org/downloads/",
  "providers-url": "/p/%package%$%hash%.json",
  "metadata-url": "/p2/%package%.json",
  "search": "https://packagist.org/search.json?q=%query%&type=%type%",
  "list": "https://packagist.org/packages/list.json",
  "providers-api": "https://packagist.org/providers/%package%.json",
  "warning": "You are using an outdated version of Composer. Composer 2.0 is now available and you should upgrade. See https://getcomposer.org/2",
  "warning-versions": "<1.10.10",
  "provider-includes": {
    "p/provider-2013$%hash%.json": {
      "sha256": "6f5245b8e32c133b033a6b94a94b83e2cff7b4e2f8cb4a95d4d072b445425efc"
    },
    "p/provider-2014$%hash%.json": {
      "sha256": "0dea2415f51bb42f302734a54712bd4fd1ca0b5a5522eef2c9c92a6604de62e1"
    },

What's Composer Mirror?

    "p/provider-2015$%hash%.json": {
      "sha256": "06445a5bb49da72f05df786b32c37dd8fa3e6018417231e86bbeb64ba1033414"
    },
    "p/provider-2016$%hash%.json": {
      "sha256": "ea39fcc510e033d534321c6c38b1177c1395714f1bd2f88b4776122329b436a9"
    },
    "p/provider-2017$%hash%.json": {
      "sha256": "b5d475e06468b9012a6abda77b95bf8c0c810caf846171d6e600b233eb32cadd"
    },
    "p/provider-2018$%hash%.json": {
      "sha256": "2d7203c374d27212c21282e88fcd3821b5ff087104e820f698fa31a4bba986a8"
    },
    "p/provider-2019$%hash%.json": {
      "sha256": "024c8665fd16738a6939297ca3e1ca3ddd09e8de25a71acfbd606d26b9db39b0"
    },
    "p/provider-2020-01$%hash%.json": {
      "sha256": "37d36a410488b0740d74779344042d5faa827446248f4900bf10fd32f5904730"
    },

What's Composer Mirror?

    "p/provider-2020-04$%hash%.json": {
      "sha256": "a1903a10335e83f0a9e6ed56db7d1caec0ba6e643623da93ba980934d3bfa9a7"
    },
    "p/provider-2020-07$%hash%.json": {
      "sha256": "4f7f0f292534aa0d5752d7dcde9c2760e692ba6d3f7b7ac364ea7b852ff08c5f"
    },
    "p/provider-2020-10$%hash%.json": {
      "sha256": "67f6fe48dd03bd2f1c37a3b691454b4ecb5ad57a14ae1b34b7da0e652274bc8e"
    },
    "p/provider-archived$%hash%.json": {
      "sha256": "674180e1623de22ac9cb0e78fd09560149e4971a1c8adc2b85df88846e2fb5cf"
    },
    "p/provider-latest$%hash%.json": {
      "sha256": "9e7775a3cdd179d774e12f88f5def97e92fc41c44a5d0b359b14bcb3c983aacd"
    }
  }
}

What's Composer Mirror?

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

Building a Composer Mirror-part1

# Application name
APP_NAME='Packagist Mirror'

# Language
APP_COUNTRY_NAME='Brazil'
APP_COUNTRY_CODE='br'

# Folder used to save the files
PUBLIC_DIR=./public

# Sync interval(show in page)
SLEEP=300
# SLEEP=0 Synchronized continuously

# Maintainer information
MAINTAINER_MIRROR='Webysther'
MAINTAINER_PROFILE='https://github.com/Webysther'
MAINTAINER_REPO='https://github.com/Webysther/packagist-mirror'
MAINTAINER_LICENSE='MIT License'

# Main mirror used to get providers
# Official - https://repo.packagist.org
# UK - https://repo-eu-uk-1.packagist.org
MAIN_MIRROR=https://repo.packagist.org

Building a Composer Mirror-part1

# Pattern to generate the package URIs.
# I.E. packages.drupal.org uses: URI_PATTERN='%s$%s.json'
URI_PATTERN='p/%s$%s.json'

# Timezone
TZ='America/Sao_Paulo'

# https://packagist.co.za
# https://mirrors.aliyun.com/composer
# https://packagist.mirrors.sjtug.sjtu.edu.cn
# https://php.cnpkg.org
# https://mirrors.cloud.tencent.com/composer
# https://packagist.jp
# https://packagist.kr
# https://packagist.com.br
# https://packagist.phpindonesia.id
DATA_MIRROR=https://packagist.co.za,https://mirrors.aliyun.com/composer,https://packagist.mirrors.sjtug.sjtu.edu.cn,https://php.cnpkg.org,https://mirrors.cloud.tencent.com/composer,https://packagist.jp,https://packagist.kr,https://packagist.com.br,https://packagist.phpindonesia.id,https://repo-eu-uk-1.packagist.org

Building a Composer Mirror-part1


# URI of your mirror
URL=packagist.com.br

# Since your mirror exists
SINCE="Q3'17"

# Max connections by mirror
MAX_CONNECTIONS=25

# Google Analytics
GOOGLE_ANALYTICS_ID=

# Google Analytics, keep this value to have in future stats about packagist-mirror worldwide coverage.
GOOGLE_ANALYTICS_MAIN_ID='UA-58875124-2'

Building a Composer Mirror-part1

After few months...

webysther/packagist-mirror

webysther/packagist-mirror

webysther/packagist-mirror

webysther/packagist-mirror

webysther/packagist-mirror

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Building a Composer Mirror-part2

Future Works

Composer V1 Issues

Why Slow?

Downloading packages is slow

Downloading packages is slow with non-parallel

Parallel Solution for Composer V1

https://github.com/hirak/prestissimo

Composer V2 Benefits

Downloading packages is with parallel

Composer V2 Benefits

Composer V2 Benefits

  • Partial offline support

    • COMPOSER_DISABLE_NETWORK=1

  • Run-time platform requirements check

    • vendor/composer/platform_check.php

  • Dry-run support for require and remove commands

  • Running as root requires confirmation

  • New repository metadata format Support

Selecting Hosting Providers Best Practices

不是主機供應商業配文

Selecting Hosting Providers Best Practices

concerned issues

  • CPU Cores

  • RAM

  • Disk Spaces

  • Network Bandwidths

  • Customer Support

  • Control Panel

  • Budget

    • NT $1000 ↓ per month

Selecting Hosting Providers Best Practices

PhotonVPS

CPU RAM Disk Spaces Network
Bandwidths
Support Control Panel Budget
(per month) (US)
2 Cores 4 GB 80 GB 100 Mbps English WHCMS $20

Selecting Hosting Providers Best Practices

PhotonVPS----Many Problems

Selecting Hosting Providers Best Practices

Imcloud VPS 天空數位

CPU RAM Disk Spaces Network
Bandwidths
Support Control Panel Budget
(per month) (US)
2 Cores 4 GB 30 GB 20 Mbps 中文 WHCMS with Customized $16

Selecting Hosting Providers Best Practices

Selecting Hosting Providers Best Practices

SimpleCloud VPS 立京資訊

CPU RAM Disk Spaces Network
Traffics
Support Control Panel Budget
(per month) (US)
2 Cores 4 GB 50 GB 1000 GB 中文 SolusVM $30 (付款可以轉帳)

Selecting Hosting Providers Best Practices

SimpleCloud VPS 立京資訊

Reinstalling Ubuntu 18.04 has been problematic

Ubuntu 18.04 Reinstalling Problem

Ubuntu 18.04 Reinstalling Problem

Selecting Hosting Providers Best Practices

SOFTSHELLWEB

CPU RAM Disk Spaces Network
Bandwidths
Support Control Panel Budget
(per month) (US)
2 Cores 4 GB 80 GB 1 Gbps English Customized $14

Lookup IP

repo.packagist.org

repo.packagist.org

PhotonVPS

天空數位

立京資訊

softshellweb

Thanks!

References

References

References

References

Any questions?