Tech from startup to enterprise
Miro Svrtan (@msvrtan)
15y PHP veteran
ZgPHP & WebCamp co-organizer
@Njuškalo for 5y
Technical analyst
Development team leader
Architect
Developer
Njuskalo.hr introduction
- online classifieds platform
- based in Zagreb, Croatia
- founded in 2007
- startup inside a large media corporation
Team organization
Small development team
- 3 backend devs
- 1 frontend dev
- small but dedicated & focused team
Platform: software
- LAMP stack
- Nginx proxy
- Sphinx search
- Memcached + Redis
Platform: code
- in house MVC PHP framework
- 2009: too many flavours of OS FWs
- 2014: IMHO loosing a battle to OS FWs
- small team vs vast community
- learning best practices
- introduction to new developers
- existing expirience
- better documentation
- tutorials/howto's
Page views in last 5y
Unique visitors in last 5y
2009 vs 2014
Visitors
425.000 (06/2009)
1.000.000 (01/2014)
Source: Gemius Audience
PageViews
25.000.000 (06/2009)
295.000.000 (01/2014)
Source: Gemius Audience
Platform: Physical location
- local data center
- low network latency -> 90% local traffic
- (possible) legal issues
Platform: Hardware
- bare metal servers
- enterprise grade storage
Bare metal servers vs cloud
- pros
- dedicated resources
- constant performance
- cons
- no quick scaling
- months instead of minutes
- hard service isolation
- add new service
Isolating a service
Resources used: 2009 vs 2014
Disk usage
< 30GB to ~10TB
Peak bandwidth
< 30Mb/s to > 450Mb/s
DB
~1GB to ~360GB
Living on the edge
- close to maximum resource utilization
- developing features with no room to spare
- concentrating on performance instead of feature
- hard & exhausting
We got room
- new shiny server is here
- vastly oversized -> easy living
- developers stop thinking about performances
Multi feature development
- test server hosts >10 Njuškalo applications
- test1.example.com
- test2.example.com
- ..
- test10.example.com
- most locations serve only 1 new feature/bug-fix
- helps to cherry pick what is ready for production
Multi feature development #2
- using git
- moved from subversion in 2010
- always working on rebased feature branch
- master branch + that feature changes
Startup
Enterprise
Becoming an enterprise
- transformation of goals/mindsets
- not something in a road map
When?
- mistakes "cost" more than
- better/longer preparation
- more testing
- additional hardware
Cost of doing errors
- not just money!
- brand name recognition
- loosing trust of users
- personal pride
Journey
- started with ~0 scaling experience
- hard to come by
- there is no manual
- listen & read what/how others did it
- outside PHP || web world too!!
Monitoring
- started with
- dstat
- gemius
- added
- cacti
- Google Analytics real time
- Graphite
- NewRelic
Autoload
- irritated with requires/includes
- different naming schemes
- before PSR-0
- parsing PHP files to generate map
- file array vs APC user cache
- after 4 years found a CRITICAL bug
- logical flaw
- production crashes
So when can you be sure your 100 lines of code are bug free?
If it worked first 10.000 times?
.. 100.000 times?
... 1.000.000 times?
... 1.000.000.000 times?
Well 8.000.000.000 times was not enough for this one :)
Tips n tricks:
Linux desktop
- development team switched to Linux desktop
- easier environment to setup
- get to know terminal
- similar to production
- reduces fear when working on production
Tips n tricks:
face 2 face discussions
- improves communication
- meetings outside office when ever possible
- less interruptions
- more relaxed atmosphere
- change of scenery
- boosts creativity & engagement
Tips n tricks:
analyze seasonality
- locate
- peaks
- low times
- predict user behavior -> scaling
- we found patterns based on
- hours in day
- day of week
- months in year
- weather!
Tips n tricks:
deployment guidelines
- not after 4pm
- avoid fridays
- major upgrades during night
Thank you
Miro Svrtan
@msvrtan