Acerca de mí

Fabrizio Balliano
@fballiano
PHP desde 1999
Magento desde 2007
magento.com/it admin
Full time Magento desde 2011
Freelance en Lanzarote                                                                                                                                                             


¿QUÉ ES VARNISH?

VARNISH:

 500% PERFORMANCE BOOST


QUÉ ES VARNISH




 LO QUIERO!




pero...

QUÉ ES VARNISH





...en detalle... 

Qué es varnish




Web application accelerator
Reverse proxy  HTTP
Cache server
Load balancer

Enorme ganancia de velocidad
Menor cargo de CPU
Menor tráfico de red
Solución full RAM

QUÉ ES VarnisH en Magento




Full Page Cache
- pre rendered HTML

Asset estáticos
- JS/CSS
- Imágenes

LISTA DE COMPRAS


  • Varnish, por supuesto
  • Mòdulo Magento (Turpentine?)
  • Nuevo servidor?
  • Configuración

ejemplo de infraestructura



PARECE BASTANCE SENCILLO





...pero...

PUNTOS CRITICOS



  • Configuración
  • Tracking de visitantes
  • Bloques dinámicos
  • SSL

CONFIGURACIÓN

Configuración


 sub generate_session {
    # generate a UUID and add `frontend=$UUID` to the Cookie header, or use SID
    # from SID URL param
    if (req.url ~ ".*[&?]SID=([^&]+).*") {
        set req.http.X-Varnish-Faked-Session = regsub(
            req.url, ".*[&?]SID=([^&]+).*", "frontend=\1");
    } else {
        C{
            char uuid_buf [50];
            generate_uuid(uuid_buf);
            static const struct gethdr_s VGC_HDR_REQ_VARNISH_FAKED_SESSION =
            { HDR_REQ, "\030X-Varnish-Faked-Session:"};
            VRT_SetHdr(ctx,
                &VGC_HDR_REQ_VARNISH_FAKED_SESSION,
                uuid_buf,
                vrt_magic_string_end
            );
        }C
    }
    if (req.http.Cookie) {
        # client sent us cookies, just not a frontend cookie. try not to blow
        # away the extra cookies
        std.collect(req.http.Cookie);
        set req.http.Cookie = req.http.X-Varnish-Faked-Session +
            "; " + req.http.Cookie;
    } else {
        set req.http.Cookie = req.http.X-Varnish-Faked-Session;
    }
}
    
sub vcl_recv {
    {{maintenance_allowed_ips}}

    # this always needs to be done so it's up at the top
    if (req.restarts == 0) {
        if (req.http.X-Forwarded-For) {
            set req.http.X-Forwarded-For =
                req.http.X-Forwarded-For + ", " + client.ip;
        } else {
            set req.http.X-Forwarded-For = client.ip;
        }
    }

    # We only deal with GET and HEAD by default
    # we test this here instead of inside the url base regex section
    # so we can disable caching for the entire site if needed
    if (!{{enable_caching}} || req.http.Authorization ||
        req.method !~ "^(GET|HEAD|OPTIONS)$" ||
        req.http.Cookie ~ "varnish_bypass={{secret_handshake}}") {
        return (pipe);
    }

    if({{send_unmodified_url}}) {
        # save the unmodified url
        set req.http.X-Varnish-Origin-Url = req.url;
    }

    # remove double slashes from the URL, for higher cache hit rate
    set req.url = regsuball(req.url, "(.*)//+(.*)", "\1/\2");

    {{normalize_encoding}}
    {{normalize_user_agent}}
    {{normalize_host}}

    # check if the request is for part of magento
    if (req.url ~ "{{url_base_regex}}") {
        # set this so Turpentine can see the request passed through Varnish
        set req.http.X-Turpentine-Secret-Handshake = "{{secret_handshake}}";
        # use the special admin backend and pipe if it's for the admin section
        if (req.url ~ "{{url_base_regex}}{{admin_frontname}}") {
            set req.backend_hint = admin;
            return (pipe);
        }
        if (req.http.Cookie ~ "\bcurrency=") {
            set req.http.X-Varnish-Currency = regsub(
                req.http.Cookie, ".*\bcurrency=([^;]*).*", "\1");
        }
        if (req.http.Cookie ~ "\bstore=") {
            set req.http.X-Varnish-Store = regsub(
                req.http.Cookie, ".*\bstore=([^;]*).*", "\1");
        }
        # looks like an ESI request, add some extra vars for further processing
        if (req.url ~ "/turpentine/esi/get(?:Block|FormKey)/") {
            set req.http.X-Varnish-Esi-Method = regsub(
                req.url, ".*/{{esi_method_param}}/(\w+)/.*", "\1");
            set req.http.X-Varnish-Esi-Access = regsub(
                req.url, ".*/{{esi_cache_type_param}}/(\w+)/.*", "\1");

            # throw a forbidden error if debugging is off and a esi block is
            # requested by the user (does not apply to ajax blocks)
            if (req.http.X-Varnish-Esi-Method == "esi" && req.esi_level == 0 &&
                    !({{debug_headers}} || client.ip ~ debug_acl)) {
                return (synth(403, "External ESI requests are not allowed"));
            }
        }
        {{allowed_hosts}}
        # no frontend cookie was sent to us AND this is not an ESI or AJAX call
        if (req.http.Cookie !~ "frontend=" && !req.http.X-Varnish-Esi-Method) {
            if (client.ip ~ crawler_acl ||
                    req.http.User-Agent ~ "^(?:{{crawler_user_agent_regex}})$") {
                # it's a crawler, give it a fake cookie
                set req.http.Cookie = "frontend=crawler-session";
            } else {
                # it's a real user, make up a new session for them
                {{generate_session}}
            }
        }
        if ({{force_cache_static}} &&
                req.url ~ ".*\.(?:{{static_extensions}})(?=\?|&|$)") {
            # don't need cookies for static assets
            unset req.http.Cookie;
            unset req.http.X-Varnish-Faked-Session;
            return (hash);
        }
        # this doesn't need a enable_url_excludes because we can be reasonably
        # certain that cron.php at least will always be in it, so it will
        # never be empty
        if (req.url ~ "{{url_base_regex}}(?:{{url_excludes}})" ||
                # user switched stores. we pipe this instead of passing below because
                # switching stores doesn't redirect (302), just acts like a link to
                # another page (200) so the Set-Cookie header would be removed
                req.url ~ "\?.*__from_store=") {
            return (pipe);
        }
        if ({{enable_get_excludes}} &&
                req.url ~ "(?:[?&](?:{{get_param_excludes}})(?=[&=]|$))") {
            # TODO: should this be pass or pipe?
            return (pass);
        }
        if (req.url ~ "[?&](utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=") {
            # Strip out Google related parameters
            set req.url = regsuball(req.url, "(?:(\?)?|&)(?:utm_source|utm_medium|utm_campaign|gclid|cx|ie|cof|siteurl)=[^&]+", "\1");
            set req.url = regsuball(req.url, "(?:(\?)&|\?$)", "\1");
        }

        if ({{enable_get_ignored}} && req.url ~ "[?&]({{get_param_ignored}})=") {
            # Strip out Ignored GET parameters
            set req.url = regsuball(req.url, "(?:(\?)?|&)(?:{{get_param_ignored}})=[^&]+", "\1");
            set req.url = regsuball(req.url, "(?:(\?)&|\?$)", "\1");
        }

        if({{send_unmodified_url}}) {
            set req.http.X-Varnish-Cache-Url = req.url;
            set req.url = req.http.X-Varnish-Origin-Url;
            unset req.http.X-Varnish-Origin-Url;
        }

        # everything else checks out, try and pull from the cache
        return (hash);
    }
    # else it's not part of magento so do default handling (doesn't help
    # things underneath magento but we can't detect that)
}

sub vcl_pipe {
    # since we're not going to do any stuff to the response we pretend the
    # request didn't pass through Varnish
    unset bereq.http.X-Turpentine-Secret-Handshake;
    set bereq.http.Connection = "close";
}

# sub vcl_pass {
#     return (pass);
# }

sub vcl_hash {

    if({{send_unmodified_url}} && req.http.X-Varnish-Cache-Url) {
        hash_data(req.http.X-Varnish-Cache-Url);
    } else {
        hash_data(req.url);
    }

    if (req.http.Host) {
        hash_data(req.http.Host);
    } else {
        hash_data(server.ip);
    }
    hash_data(req.http.Ssl-Offloaded);
    if (req.http.X-Normalized-User-Agent) {
        hash_data(req.http.X-Normalized-User-Agent);
    }
    if (req.http.Accept-Encoding) {
        # make sure we give back the right encoding
        hash_data(req.http.Accept-Encoding);
    }
    if (req.http.X-Varnish-Store || req.http.X-Varnish-Currency) {
        # make sure data is for the right store and currency based on the *store*
        # and *currency* cookies
        hash_data("s=" + req.http.X-Varnish-Store + "&c=" + req.http.X-Varnish-Currency);
    }

    if (req.http.X-Varnish-Esi-Access == "private" &&
            req.http.Cookie ~ "frontend=") {
        hash_data(regsub(req.http.Cookie, "^.*?frontend=([^;]*);*.*$", "\1"));
        {{advanced_session_validation}}

    }
    return (lookup);
}

sub vcl_hit {
    # this seems to cause cache object contention issues so removed for now
    # TODO: use obj.hits % something maybe
    # if (obj.hits > 0) {
    #     set obj.ttl = obj.ttl + {{lru_factor}}s;
    # }
}

# sub vcl_miss {
#     return (fetch);
# }

sub vcl_backend_response {
    # set the grace period
    set beresp.grace = {{grace_period}}s;

    # Store the URL in the response object, to be able to do lurker friendly bans later
    set beresp.http.X-Varnish-Host = bereq.http.host;
    set beresp.http.X-Varnish-URL = bereq.url;

    # if it's part of magento...
    if (bereq.url ~ "{{url_base_regex}}") {
        # we handle the Vary stuff ourselves for now, we'll want to actually
        # use this eventually for compatibility with downstream proxies
        # TODO: only remove the User-Agent field from this if it exists
        unset beresp.http.Vary;
        # we pretty much always want to do this
        set beresp.do_gzip = true;

        if (beresp.status != 200 && beresp.status != 404) {
            # pass anything that isn't a 200 or 404
            set beresp.ttl = {{grace_period}}s;
            set beresp.uncacheable = true;
            return (deliver);
        } else {
            # if Magento sent us a Set-Cookie header, we'll put it somewhere
            # else for now
            if (beresp.http.Set-Cookie) {
                set beresp.http.X-Varnish-Set-Cookie = beresp.http.Set-Cookie;
                unset beresp.http.Set-Cookie;
            }
            # we'll set our own cache headers if we need them
            unset beresp.http.Cache-Control;
            unset beresp.http.Expires;
            unset beresp.http.Pragma;
            unset beresp.http.Cache;
            unset beresp.http.Age;

            if (beresp.http.X-Turpentine-Esi == "1") {
                set beresp.do_esi = true;
            }
            if (beresp.http.X-Turpentine-Cache == "0") {
                set beresp.ttl = {{grace_period}}s;
                set beresp.uncacheable = true;
                return (deliver);
            } else {
                if ({{force_cache_static}} &&
                        bereq.url ~ ".*\.(?:{{static_extensions}})(?=\?|&|$)") {
                    # it's a static asset
                    set beresp.ttl = {{static_ttl}}s;
                    set beresp.http.Cache-Control = "max-age={{static_ttl}}";
                } elseif (bereq.http.X-Varnish-Esi-Method) {
                    # it's a ESI request
                    if (bereq.http.X-Varnish-Esi-Access == "private" &&
                            bereq.http.Cookie ~ "frontend=") {
                        # set this header so we can ban by session from Turpentine
                        set beresp.http.X-Varnish-Session = regsub(bereq.http.Cookie,
                            "^.*?frontend=([^;]*);*.*$", "\1");
                    }
                    if (bereq.http.X-Varnish-Esi-Method == "ajax" &&
                            bereq.http.X-Varnish-Esi-Access == "public") {
                        set beresp.http.Cache-Control = "max-age=" + regsub(
                            bereq.url, ".*/{{esi_ttl_param}}/(\d+)/.*", "\1");
                    }
                    set beresp.ttl = std.duration(
                        regsub(
                            bereq.url, ".*/{{esi_ttl_param}}/(\d+)/.*", "\1s"),
                        300s);
                    if (beresp.ttl == 0s) {
                        # this is probably faster than bothering with 0 ttl
                        # cache objects
                        set beresp.ttl = {{grace_period}}s;
                        set beresp.uncacheable = true;
                        return (deliver);
                    }
                } else {
                    {{url_ttls}}
                }
            }
        }
        # we've done what we need to, send to the client
        return (deliver);
    }
    # else it's not part of Magento so use the default Varnish handling
}

{{vcl_synth}}

sub vcl_deliver {
    if (req.http.X-Varnish-Faked-Session) {
        # need to set the set-cookie header since we just made it out of thin air
        {{generate_session_expires}}
        set resp.http.Set-Cookie = req.http.X-Varnish-Faked-Session +
            "; expires=" + resp.http.X-Varnish-Cookie-Expires + "; path=/";
        if (req.http.Host) {
            if (req.http.User-Agent ~ "^(?:{{crawler_user_agent_regex}})$") {
                # it's a crawler, no need to share cookies
                set resp.http.Set-Cookie = resp.http.Set-Cookie +
                "; domain=" + regsub(req.http.Host, ":\d+$", "");
            } else {
                # it's a real user, allow sharing of cookies between stores
                if(req.http.Host ~ "{{normalize_cookie_regex}}") {
                    set resp.http.Set-Cookie = resp.http.Set-Cookie +
                    "; domain={{normalize_cookie_target}}";
                } else {
                    set resp.http.Set-Cookie = resp.http.Set-Cookie +
                    "; domain=" + regsub(req.http.Host, ":\d+$", "");
                }
            }
        }
        set resp.http.Set-Cookie = resp.http.Set-Cookie + "; httponly";
        unset resp.http.X-Varnish-Cookie-Expires;
    }
    if (req.http.X-Varnish-Esi-Method == "ajax" && req.http.X-Varnish-Esi-Access == "private") {
        set resp.http.Cache-Control = "no-cache";
    }
    if ({{debug_headers}} || client.ip ~ debug_acl) {
        # debugging is on, give some extra info
        set resp.http.X-Varnish-Hits = obj.hits;
        set resp.http.X-Varnish-Esi-Method = req.http.X-Varnish-Esi-Method;
        set resp.http.X-Varnish-Esi-Access = req.http.X-Varnish-Esi-Access;
        set resp.http.X-Varnish-Currency = req.http.X-Varnish-Currency;
        set resp.http.X-Varnish-Store = req.http.X-Varnish-Store;
    } else {
        # remove Varnish fingerprints
        unset resp.http.X-Varnish;
        unset resp.http.Via;
        unset resp.http.X-Powered-By;
        unset resp.http.Server;
        unset resp.http.X-Turpentine-Cache;
        unset resp.http.X-Turpentine-Esi;
        unset resp.http.X-Turpentine-Flush-Events;
        unset resp.http.X-Turpentine-Block;
        unset resp.http.X-Varnish-Session;
        unset resp.http.X-Varnish-Host;
        unset resp.http.X-Varnish-URL;
        # this header indicates the session that originally generated a cached
        # page. it *must* not be sent to a client in production with lax
        # session validation or that session can be hijacked
        unset resp.http.X-Varnish-Set-Cookie;
    }
}
http://tinyurl.com/pxnyaht 

VCL FLOW


TRACKING DE LOS VISITANTES

ruta visitantes
pagina no en cache



ruta visitantes 
pagina en cache



Lo que pasa es que...




los servidores web
no son conscientes
de las visitas
a las paginas en cache

Quien hace server side tracking?


Todos los servicios de tracking de visitantes
son hechos en javascript

Alguien dices Google Analytics?

MAGENTO!




Report a backend
Productos recién vistos
Customer segmentation

BLOQUES DINÁMICOS

Bloques dinámicos



Tipos de Bloques dinámicos

ESI: Edge Side Includes
  • Invisibles a los clientes
  • Antes page load
  • Pueden ser en cache
  • Todo el web/una pagina
  • Un usuario/todos
  • NO COOKIES!
  • Pocos datos
AJAX
  • Visibles a los clientes 
  • Después page load
  • No en cached
  • COOKIES!
  • Muchos datos                              

    Flash messages!



    Flash messages:  Ampliarlos
    Y SOLUCIONAR LOS PROBLEMAS
    DE TRACKING DE VISITANTES



    SSL

    VARNISH = NO SSL






    Pero lo necesito


    añadir un servidor web
    (pound, nginx, apache, haproxy)
     para responder en la puerta 443
    y hacer forward a varnish
    <virtualhost *:443>
        ServerName www.example.com
    
        ProxyPreserveHost On
        ProxyPass / http://127.0.0.1:80/
        RequestHeader set X-Forwarded-Port "443"
        RequestHeader set X-Forwarded-Proto "https"
    
        SSLEngine On
        SSLCertificateFile /etc/apache2/ssl/example.com.crt
        SSLCertificateKeyFile /etc/apache2/ssl/example.com.key
        SSLCertificateChainFile /etc/apache2/ssl/example.com.chain
    </virtualhost>        

    CONCLUSIONES

    Magento + Turpentine + Varnish =




    muy potente pero muy complejo para conducir
    necesita practicar y siempre tener cuidado

    ¡Muchas gracias!



    Varnish: pro y contras, beneficios y limitaciones.

    By Fabrizio Balliano

    Varnish: pro y contras, beneficios y limitaciones.

    Varnish es una herramienta poderosa, puede transformar tu comercio electrónico en un Ferrari pero recuerda, para conducir un Ferrari tiene que ser un piloto de Formula1. En esta charla vamos a entender lo que es Varnish, que puede darnos y cuáles son sus limitaciones, junto con algunos trucos para evitarlas.

    • 941

    More from Fabrizio Balliano