Getting Dirty With Realm.io

@attheodo

What is Realm?

  • Realm is a truly mobile database.

  • Realm is a replacement for SQLite, Core Data, JSON/Property List files.

  • Realm is easy to use, fast, cross-platform and well supported.

1

Realm is a Mobile Database

  • Models!

    • Relationships (one-to-one, one-to-many)

  • CRUD!

    • Create

    • Read

    • Update

    • Delete

  • Queries!

  • Migrations!

  • ACID Transactions

Realm is a Replacement for

  • SQLite

    • FMDB is cool...

    • ... said someone in 2011.

  • Core Data

    • Learning curve anyone?

    • ORM on top of SQLite

    • Complex API

  • Offline JSON/Plist files

    • Ok for small and simple transactions

    • Limited Flexibility

    • Too much I/O

    • Thread-safety? ACID?

Realm is easy to use

  • Realm is not an ORM on top of SQLite/CoreData/x

    • Custom C++ core with bit-packing, caching, vectorization and zero-copy architecture.

  • It has it's own conventions and data-structures that are easy to grasp and use.

    • Basically just 3 classes (Object, Arrays and Realms)

  • You can use Realm models convention to write all your app models if you want.

  • More later...

Realm is Fast

Realm is Fast

Realm is Fast

Realm is Cross-Platform

  • Objective-C, Swift

  • iOS, watchOS, tvOS

  • React-Native




  • Android

Realm Internals

 

  • Realm is truly innovative

    • MVCC

    • Native Links

    • Crash Safety

    • Zero Copy

    • True Lazy-Loading

  • Built from scratch with mobile platforms and limitations in mind

    • ​Why aim to be massively distributed?

    • Why aim to be  shardable across instances?

    • Why be low latency over connection sockets?

Realm Internals

 

  • Realm is objects all the way down.

  • Most existing solutions are ORMs on top of something.

  • ORMs abstract what's going underneath

    • Objects become records/rows in tables with foreign keys and primary keys.

    • The abstraction kind of falls apart because you need to start traversing relationships using expensive operations.

Realm Internals

  • When you init a Realm, you're already "connected" to the database

    • Actually a fraction of the database is being memory mapped, there's no socket whatsoever.

  • As soon as you add an object (record) in Realm, it becomes an accessor.

    • Once you start reading properties from it, you are not accessing your iVars!

    • You access raw database values avoiding lots of memory copying.

Realm Internals (B+ Trees, Native Links)

Writer

Reader

HEAD
HEAD~1
  • The file structure is just a tree of links.

  • Raw pointers to objects.

  • Same if you're querying for an Int, String or relationship!

Realm Internals (Crash Safety)

HEAD
HEAD~1
  • As you make changes, you're forking the tree.

  • The top level pointer "HEAD" is always pointing to the latest non-corrupt tree.

  • When Realm verifies the newest write transaction (fsync()) it move's the top level pointer and get a new "official version"

Realm Internals (Zero Copy)

How most ORMs deal with data

  • Data sits on disk.

  • You access a CoreData model's property:

    • Translate request into a set of SQL statements.

    • Create database connection.

    • Perform the SQL query.

    • Read all the data from the rows that match the query and bring it to memory.

    • Deserialize from the disk format to something you can represent in memory (align bits).

    • Convert to the language type.

Realm Internals (Zero Copy)

How Realm does it

  • The data is still on disk but it's memory mapped.

    • You can access any offset in the file as if it was in memory, although it's not. It's in virtual memory. 

  • Objects all the way down, everything is word-aligned (limitations)

  • The core file format is readable in memory without requiring deseriliazation (core Realm principle)

    • No copy to memory!

  • All you need is calculate the offset of the data to read in the mmap, read the value from offset up to offset + propery.length and return the raw value from the property access!

Realm Internals (True Lazy-Loading)

  • Hardware limitations don't allow to read just 1 bit form disk

    • Even if you want a Boolean property, you need to load the disk's page size, nothing smaller than that.

  • Most databases store things on a horizontal, linear level.

    • When you want a property from SQLite, you still have to load the entire row because it's stored contiguously in the file.

  • Realm stores properties contiguously linked at the vertical level in vectors (columns vs rows)

    • An object's property is never in an iVar unless you actually try to read it or modify it.

Installation

You guessed it. It's a Cocoapod.

pod 'RealmSwift'

Realm Models

  • Supported Property Types

    • Bool

    • Int8, Int16, Int32, Int64

      • Storing optional numbers is done using RealmOptional

    • Double, Float

    • String (can be optional)

    • NSDate (truncated to the second, can be optional)

    • NSData (can be optional)

Realm Models

import RealmSwift

class User: Object {

    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)

}

Auto-incrementing id?

NOPE.

Realm Models

import RealmSwift

class User: Object {

    dynamic var id = ""

    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)

    override static func primaryKey() -> String? {
        return "id"
    }

}

Or just map the id to your server database's id.

Realm Models

import RealmSwift

class Post: Object {

    dynamic var id = NSUUID().UUIString

    dynamic var postDescription = ""
    dynamic var postPhotoURL = ""
    dynamic var numOfLikes = 0
    dynamic var createdAt = NSDate(timeIntervalSince1970: 1)

    override static func primaryKey() -> String? {
        return "id"
    }

}

Realm Models - Relationships

import RealmSwift

class User: Object {

    dynamic var id = ""

    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)

    let posts = List<Post>()

    override static func primaryKey() -> String? {
        return "id"
    }

}

One-To-Many Relationship

Realm Models

import RealmSwift

class Post: Object {

    dynamic var id = ""

    dynamic var postDescription = ""
    dynamic var postPhotoURL = ""
    dynamic var numOfLikes = 0
    dynamic var createdAt = NSDate(timeIntervalSince1970: 1)

    var postOwners: [User] {
        return linkingObject(User.self, forProperty: "posts")
    }

    override static func primaryKey() -> String? {
        return "id"
    }

}

Inverse Relationship

Realm Models - Ignored Properties

import RealmSwift

class User: Object {

    dynamic var id = ""

    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)

    let posts = List<Post>()

    override static func primaryKey() -> String? {
        return "id"
    }

    override static func ignoredProperties() -> [String] {
        return ["nickname"]
    }

}

Realm Models - Indexes

import RealmSwift

class User: Object {

    dynamic var id = ""

    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)

    let posts = List<Post>()

    override static func primaryKey() -> String? {
        return "id"
    }

    override static func indexedProperties() -> [String] {
        return ["id", "firstName"]
    }

}

Greatly speeds up queries at

the cost of slower insertions.

Creating Realm Objects

Way 1: Object Composition

import RealmSwift

var user = User()

user.id = NSUUID().UUIString
user.firstname = "Thanos"
user.lastname = "Theodoridis"
user.email = "at@atworks.gr"

Creating Realm Objects

Way 2: Init from Dictionary

import RealmSwift

var user = User(value: [
    "id": NSUUID().UUIString,
    "firstname": "Thanos", 
    "lastname": "Theodoridis", 
    "email": "at@atworks.gr"
])

Creating Realm Objects

Way 3: Init from Array

import RealmSwift

var user = User(value: [
    "Thanos", 
    "Theodoridis", 
    "at@atworks.gr"
])

Creating (Nested) Realm Objects

var post1 = Post(value: [
    "postDescription": "Look, my dog! #dog #igers", 
    "postURL": "http://lala.com/dog.jpg", 
])

var post2 = Post(value: [
    "postDescription": "Look, my cat! #cute", 
    "postURL": "http://lala.com/cat.jpg", 
])

var user = User(value: [
    "id": NSUUID().UUIString,
    "firstname": "Thanos", 
    "lastname": "Theodoridis", 
    "email": "at@atworks.gr",
    "posts": [post1, post2]
])

Persisting Realm Objects

let realm = try! Realm()

var user = User(value: [
    "firstname": "Thanos", 
    "lastname": "Theodoridis", 
    "email": "at@atworks.gr",
    "posts": [post1, post2]
])

try! realm.write {
    realm.add(user)
}

Updating Realm Objects

let realm = try! Realm()

user.lastname = "Papadopoulos"

try! realm.write {
    realm.add(user, update: true)
}
  • This will work only if the User object has a primary key.

    • If not, you need to enclose `user.lastname` in a `write()` block.

  • If you omit `update: true` and an object with that key already exists, you get an exception (if unhandled, you crash)

Queries

  • All queries return a `Results` instance with a collection of Objects.

  • All queries are lazy. 

    • Data is only read when the properties are accessed. 

  • Results are not copies. 

    • Modifying a Results object within a write transaction will persist to disk. 

  • Query execution is deferred until the results are used.

  • Once a query has been executed, the Results object is up to date with any other changes in Realm.

Queries

  • The equivalent of `SELECT * FROM 'USERS'`

let realm = try! Realm()

let users = realm.objects(User)

Complex Queries? 

NSPredicate()

Queries for Relationships

let realm = try! Realm()

// Forward relationship
let user = realm.objects(User).first
let posts = user.posts

// Inverse (backward) relationship
let post = realm.objects(Post).first
let postUser = post.postOwners.first

Queries using NSPredicate()

let realm = try! Realm()

// Get a user with a matching id
let predicate = NSPredicate(format: "id == %@", id)
let user1 = realm.objects(User).filter(predicate)

// Get users with GMail accounts
predicate = NSPredicate(format: "email ENDSWITH %@", "gmail.com")
let users = realm.objects(User).filter(predicate)

// Get top active users
predicate = NSPredicate(format: "posts.@count > 100")
let activeUsers = realm.objects(User).filter(predicate)

// Get users registered before Christmas
predicate = NSPredicate(format: "date <= %@", christmasDate)

The Realm Database

  • Realm() gets you the default Realm database.

    • A file named `default.realm` in /Documents

var config = Realm.Configuration()

config.path = NSURL.fileURLWithPath(config.path!)
                .URLByDeletingLastPathComponent?
                .URLByAppendingPathComponent("NewName.realm")
                .path

// set tihs as the config for the default realm
Realm.Configuration.defaultConfiguration = config

The Realm Database

  • But you can use as many different Realms as you want!

let config = Realm.Configuration(

    path: myPathToBackupRealm.path
    readOnly: false

)

// Default Realm (default.realm)
let realm = try! Realm()

// Backup Realm (backup.realm)
let otherRealm = try! Realm(configuration: config)

The Realm Database

  • You can also have in-memory Realms

    • Cool for caching layers

let memoryCache = try! Realm(configuration:
    Realm.Configuration(inMemoryIdentifier: "com.me.cache")
)
  • Beware of in-memory instances going out of scope with no references. 

    • Garbage collector will wipe your data off

  • Keep a strong reference to your in-memory Realms.

The Realm Database

  • You can also bundle prepopulated Realms with your app.

  • In the code where you generate the prepopulated Realm, make sure to "compact" it:

Realm().writeCopyToPath(_:encryptionKey:)
  • Drag and drop and add it in "Copy Bundle Resources" of your build phase.

  • If you need to mutate this Realm, your first need to copy it to your application's Documents directory. Bundles are read-only.

Thread Safety

  • One golden rule:

let realm = try! Realm()

for each thread

  • Realm state for each thread will be based off the most recent confirmed transaction commit and will remain there until that version is refreshed.

    • States are automatically refreshed at the start of every NSRunLoop iteration.

    • Background threads that do not have run loops must explicitly call Realm.refresh()

Notifications

  • Two options for monitoring updates on Realm objects:

    • Collection Notifications

    • KVO

let userModelNotificationToken: NotificationToken!

userModelNotificationToken = realm.objects(User).addNotificationBlock {
    
    notification, realm in

        usersTabelView.reloadData()

}

// When no longer needed
userModelNotificationToken.stop()

Migrations In Realm

  • Data models change...

  • Realm models are parsed and validated during compile-time.

  • But, if Realm sees a mismatch in models and the database during run-time, it throws an exception and doesn't let you read or write to the db.

  • ... Unless you run a migration!

Migrations In Realm

// v1 Model
class User: Object {

    dynamic var id = ""

    dynamic var firstname = ""
    dynamic var lastname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)

}

// v2 Model
class User: Object {

    dynamic var id = ""

    dynamic var fullname = ""
    dynamic var nickname: String? = nil
    dynamic var email = ""
    dynamic var password = ""
    dynamic var registrationDate = NSDate(timeIntervalSince1970: 1)

}

Migrations In Realm

// application(application:didFinishLaunchingWithOptions:)

Realm.Configuration.defaultConfiguration = Realm.Configuration(
    
    // Increment the db schema version
    schemaVersion: 1,
    
    // Perform the migration
    migrationBlock: { migration, oldSchemaVersion in
    
        if (oldSchemaVersion < 1) {
        
            migration.enumerate(User.className()) { oldObject, newObject in
                
                let firstname = oldObject!["firstname"] as! String
                let lastname = oldObject!["lastname"] as! String
                newObject!["fullname"] = "\(fistname) \(lastname)"
            
            }
        }
    }
    
)

Encrypted Realms

  • Realm supports encrypting the db file on disk with AES-256+SHA2 and 64byte keys

let key = NSMutableData(length: 64)!

// Generate random key
SecRandomCopyBytes(
    kSecRandomDefault, 
    key.length,
    UnsafeMutablePointer<UInt8>(key.mutableBytes)
)

// Save it securely in keychain

// Always pass it in default Realm configuration
// to be able to use encrypted Realm file
Realm.Configuration.defaultConfiguration.encryptionKey = key

Realm Browser

print(Realm.Configuration.defaultConfiguration.path!)
// or
(lldb) po Realm.defaultPath 

Questions?

@attheodo

http://attheo.do

Getting Dirty With Realm.io

By Thanos Theodoridis

Getting Dirty With Realm.io

  • 475