Email address validation, what the point?

Increasing delivery rates

Improving your email deliverability by removing all invalid email addresses from your list.

Maintains a high sender reputation

 

Malibox providers take a lot of metrics to check your sender reputation, one of them is mailing to unknown users.

Increasing conversion rate

 

More emails that arrive to inboxes means higher opens and clicks, and better performance overall.

Email Validation Methods

Regex validation
MX validation
SMTP validation

Regex Validation

Checking the email address via regex pattern

Regex Validation

TYPICAL_EMAIL = /(?=\A.{6,255}\z)(#{USER})@(#{DOMAIN})/
  • should be one character or more
  • may contain any word character, case insensitive
  • may contain dots, hyphens, but not as first character
USER = /\A([a-z0-9]+[\w|\-|\.|\+]*/i
DOMAIN = /[a-z0-9]+([\-\.]{1}[a-z0-9]+)*\.[a-z]{2,63}/i
  • should start from letter or number
  • may contain dots, hyphens
  • TLD should contain only letters and should be between 2 and 63 characters
  • case insensitive

Not follow RFC 5322!

MX Validation

Checking the availability of the email address domain using DNS MX records

MX Validation

Domain MX lookup

MX records
resolver

CNAME records
resolver

A record
resolver

validation fails

false

false

false

true

true

true

validation

successful

MX Validation - step 1

 

MX records
resolver

 

 

Doesn't Null MX exist?

RFC 7505

 

MX records exist?

RFC 5321

$ dig example.com MX

example.com. IN MX 0 mail.example.com.
mail.example.com. IN A 127.0.0.1
mail.example.com. IN A 127.0.0.2
$ dig example.com MX

example.com. IN MX 0 .

Domain name

Transform each
MX record to
host address(es)

true

true

false

validation fails

false

CNAME records
resolver

[127.0.0.1, 127.0.0.2]

validation successful

MX Validation - step 2

 

CNAME records
resolver

 

 

Do CNAME records
exist?

RFC 5321

$ dig mail.example.com CNAME

mail.example.com.    1423 IN CNAME mail.somedomain.net.
mail.somedomain.net. 6888 IN CNAME mail.nextdomain.com.
mail.nextdomain.com. 347  IN A     127.0.0.1

true

true

false

false

A record
resolver

 

MX records
resolver

 

 

Last CNAME host from each CNAME record

[127.0.0.1, 127.0.0.2]

validation

successful

MX Validation - step 3

 

A record
resolver

 

 

Do A records
exist?

 

$ dig mail.example.com A

mail.example.com. 347  IN A     127.0.0.1
mail.example.com. 355  IN A     127.0.0.2

true

false

[127.0.0.1]

validation fails

 

Host from first
A record

RFC 5321

validation successful

SMTP Validation

Checking real existence of email account on a mail server

SMTP Validation

Each mail
server

validation

fails

false

false

true

true

validation

successful

Is the 25-port
opened?

Run
SMTP-session

false

Next
mail-server exists?

true

SMTP Validation - session

validation fails

true

true

validation successful

HELO/EHLO

false

MAILFROM

RCPTTO

Open session

$ nc server.example.com 25

220 server.example.com
$ HELO yourdomain.com

250 server.example.com
$ MAIL FROM: <email@yourdomain.com>

250 2.1.0 <email@yourdomain.com> ok
$ RCPT TO: <target_email@domain.com>

250 2.1.5 <target_email@domain.com> recipient ok

true

false

false

false

$ RCPT TO: <target_email@domain.com>

550 5.7.1 No such user!

should be a real

host address should has a ptr
record (rDNS) to HELO host

$ dig -x 127.0.1.1 # your host WAN IP

127.0.0.1.in-addr.arpa. 7194 IN PTR yourdomain.com.

Truemail

Configurable framework agnostic plain Ruby email validator/verifier.

https://truemail-rb.org/truemail-gem

Truemail

  • Configurable validator
  • Zero runtime dependencies
  • Has simple SMTP debugger

Why?

Truemail - configuration

Truemail.configure do |config|
  config.verifier_email = 'verifier@example.com'

  config.verifier_domain = 'somedomain.com'

  config.email_pattern = /regex_pattern/

  config.connection_timeout = 1
  config.connection_attempts = 1
  config.response_timeout = 1

  config.validation_type_for = { 'somedomain.com' => :mx, 'otherdomain.org' => :regex }

  config.smtp_safe_check = true
  config.smtp_error_body_pattern = /regex_pattern/
end

Truemail - host audit

Truemail.host_audit
# Everything is good
=> #<Truemail::Auditor:0x00005580df358828
   @result=
     #<struct Truemail::Auditor::Result
       warnings={}>>

# Has PTR warning
=> #<Truemail::Auditor:0x00005580df358828
   @result=
     #<struct Truemail::Auditor::Result
       warnings=
         {:ptr=>"ptr record does not reference to current verifier domain"}>>

Truemail gem allows performing an audit of the host in which runs. Only PTR record audit performs for today.

It will help to check:

  • PTR existence
  • PTR reference
  • Reverse trace

Truemail - Regex validation

Truemail.validate('email@example.com', with: :regex)

=> #<Truemail::Validator:0x000055590cc9bdb8
  @result=
    #<struct Truemail::Validator::Result
      success=true, email="email@example.com",
      domain=nil,
      mail_servers=[],
      errors={},
      smtp_debug=nil>,
  @validation_type=:regex>

By default this validation not performs strictly following RFC 5322 standard, so you can override Truemail default regex pattern if you want.

Truemail - MX validation

Truemail.validate('email@example.com', with: :mx)

=> #<Truemail::Validator:0x000055590c9c1c50
  @result=
    #<struct Truemail::Validator::Result
      success=true,
      email="email@example.com",
      domain="example.com",
      mail_servers=["127.0.1.1", "127.0.1.2"],
      errors={},
      smtp_debug=nil>,
  @validation_type=:mx>

Truemail MX validator not performs strict compliance of the RFC 5321 standard for best validation outcome. Iteration will be processed for this case too:

Regex
validation

MX
validation

MX 0 unresolvable.example.com.
MX 10 healthy1.example.com.
MX 20 healthy2.example.com.

Truemail - SMTP validation

Regex
validation

MX
validation

SMTP
validation

If total count of MX servers is equal to one, Truemail::Smtp validator will use value from Configuration.connection_attempts.

 

Also you don't need pass with-parameter to use this validation.

Truemail.validate('email@example.com')
# the same, no need to do it!
Truemail.validate('email@example.com', with: :smtp)

Truemail - SMTP validation

Truemail.validate('email@example.com')

# Successful SMTP validation
=> #<Truemail::Validator:0x000055590c4dc118
  @result=
    #<struct Truemail::Validator::Result
      success=true,
      email="email@example.com",
      domain="example.com",
      mail_servers=["127.0.1.1", "127.0.1.2"],
      errors={},
      smtp_debug=nil>,
  @validation_type=:smtp>

With default settings, smtp_safe_check = false

Truemail - SMTP validation

# SMTP validation failed
=> #<Truemail::Validator:0x0000000002d5cee0
    @result=
      #<struct Truemail::Validator::Result
        success=false,
        email="email@example.com",
        domain="example.com",
        mail_servers=["127.0.1.1", "127.0.1.2"],
        errors={:smtp=>"smtp error"},
        smtp_debug=
          [#<Truemail::Validate::Smtp::Request:0x0000000002d49b10
            @configuration=#<Truemail::Configuration:0x0000000002d49930>,
            @email="email@example.com",
            @host="127.0.1.1",
            @attempts=nil,
            @response=
              #<struct Truemail::Validate::Smtp::Response
                port_opened=true,
                connection=true,
                helo=
                  #<Net::SMTP::Response:0x0000000002d5aca8
                    @status="250",
                    @string="250 127.0.1.1 Hello example.com\n">,
                mailfrom=
                  #<Net::SMTP::Response:0x0000000002d5a618
                    @status="250",
                    @string="250 OK\n">,
                rcptto=false,
                errors={:rcptto=>"550 User not found\n"}>>]>,
    @validation_type=:smtp>

With default settings,

smtp_safe_check = false

Truemail - SMTP validation

Truemail.validate('email@example.com')

# Successful SMTP validation
=> #<Truemail::Validator:0x0000000002ca2c70
    @result=
      #<struct Truemail::Validator::Result
        success=true,
        email="email@example.com",
        domain="example.com",
        mail_servers=["127.0.1.1", "127.0.1.2"],
        errors={},
        smtp_debug=
          [#<Truemail::Validate::Smtp::Request:0x0000000002c95d40
              @configuration=#<Truemail::Configuration:0x0000000002c95b38>
              @email="email@example.com",
              @host="127.0.1.1",
              @attempts=nil,
              @response=
                #<struct Truemail::Validate::Smtp::Response
                  port_opened=true,
                  connection=false,
                  helo=
                    #<Net::SMTP::Response:0x0000000002c934c8
                    @status="250",
                    @string="250 127.0.1.1\n">,
                  mailfrom=false,
                  rcptto=nil,
                  errors={:mailfrom=>"554 5.7.1 Client host blocked\n", :connection=>"server dropped connection after response"}>>,]>,
    @validation_type=:smtp>

With smtp_safe_check = true

It will be helpful if SMTP server does not return an exact answer that the email does not exist.

Truemail - SMTP validation

# SMTP validation failed
=> #<Truemail::Validator:0x0000000002d5cee0
   @result=
    #<struct Truemail::Validator::Result
      success=false,
      email="email@example.com",
      domain="example.com",
      mail_servers=["127.0.1.1", "127.0.1.2"],
      errors={:smtp=>"smtp error"},
      smtp_debug=
        [#<Truemail::Validate::Smtp::Request:0x0000000002d49b10
          @configuration=
            #<Truemail::Configuration:0x0000000002d49930
              @connection_timeout=2,
              @email_pattern=/regex_pattern/,
              @response_timeout=2,
              @connection_attempts=2,
              @smtp_safe_check=true,
              @validation_type_by_domain={},
              @verifier_domain="example.com",
              @verifier_email="verifier@example.com">,
          @email="email@example.com",
          @host="127.0.1.1",
          @attempts=nil,
          @response=
            #<struct Truemail::Validate::Smtp::Response
              port_opened=true,
              connection=true,
              helo=
              #<Net::SMTP::Response:0x0000000002d5aca8
                @status="250",
                @string="250 127.0.1.1 Hello example.com\n">,
              mailfrom=#<Net::SMTP::Response:0x0000000002d5a618 @status="250", @string="250 OK\n">,
              rcptto=false,
              errors={:rcptto=>"550 User not found\n"}>>]>,
    @validation_type=:smtp>

With smtp_safe_check = true

Truemail.configure do |c|
  c.smtp_error_body_pattern = /regex/
end

Configurable option:

# Default regex SMTP error body pattern
/(?=.*550)(?=.*(user|account|customer|mailbox)).*/i

Truemail

https://github.com/rubygarage/truemail

Bug reports and pull requests are welcome on GitHub:

The End

Email validation

By Vladislav Trotsenko

Made with Slides.com

Email validation

Email validation methods, Ruby implementation.

  • 3,585

More from Vladislav Trotsenko