Rewriting an Imageboard

BEGINNER ~ INTERMEDIATE

Suitable for Rails beginners with around 3 months of experience
A Bit of JavaScript
BBS/PTT ASCII art

老生聽到膩的老生常談

2015.rubyconf.tw

FTW CONTENT RATING

lulalala

Ruby/Rails developer at

Github: lulalala
Twitter: lulalala_it

likes anime, manga, games and drawing

When
I was a Nooblet

I was not motivated to practice any more

(●；－_－)●

So I wrote an imageboard

Futaba Channel (2chan)

4chan

Komica

http://board.lulalala.com/cat

Anonymous
Simple UI

What's so fascinating about it?

So I started Mei

40 commits
In two months
Hosted by DotCloud

(the guys who made Docker)

and then...

梅

and I forgot about mei...

I got a REAL job ( ￣ c￣)y▂ξ

Back and start again

Ruby should demonstrate its power against the PHP conterparts
o(￣皿￣///)
I don't like the current image board,
the UI really needs some improvement.
I want to see if my experience can make a difference.

Let's start!

topics

title

bumped_at

posts

id
topic_id(fk)

content

has_many

Model

Topic

Post

What did I change?

1. Multi-tenancy
2. Form Object
3. Use Cells to replace Partial
4. Avoid Form Re-render

Multi-board support

(Maybe we can call it multi-tenancy)

多住戶架構

My Rails imageboard takes around 100+ mb of memory

A PHP imageboard takes around 50+ mb of memory

So we might as well host more boards using one Ruby process

Steps

Create a board model & link topics to board
Manage board specific settings
???
PROFIT!!!

Settings

# I use SettingsLogic
def config
  @cascaded_config ||= AppSetting.board.dup.tap do |c|
    if board_config = super
      c.merge!(board_config)
    end
  end
end

Have a config :text column

serialize :config

And then merge over default settings

# Global setting
board:
  pagination:
    per_page: 5
    max_page: 10

# Board setting
# in config column
pagination:
  per_page: 3

@board = Board.find(52)
puts @board.config.pagination.per_page
#> 3
puts @board.config.pagination.max_page
#> 10

another way to save memory

Put everything under one process

Use multi-threaded servers
(e.g. puma)
Use sucker_punch for delayed jobs

Consider multi-tenancy!

moral of the rewrite #1

Make Topic & Post look like a single model using
Form Object

TopicController#create
(for creating new topic)

PostController#create
(for replying)

So there are two forms

but some logics still look VERY similar

Topic

Post

<%= form_for(@topic, format:'json', remote:true) do |f| %>
  <%= f.hidden_field :board_id %>
  <%= f.fields_for :posts, @post do |pf| %>
    <div class="field">
      <%= pf.label :author %>
      <%= pf.text_field :author %>
    </div>
    <div class="field">
      <%= pf.label :email %>
      <%= pf.text_field :email %>
    </div>
    <%= render_cell :post, :image_upload_control, pf %>
    <%= pf.label :content %>
    <%= pf.text_area :content %>
  <% end %>
  <div class="actions">
    <%= f.submit %>
  </div>
<% end %>

<%= form_for(@post, format:'json', remote:true) do |f| %>
  <%= f.hidden_field :topic_id %>

  <div class="field">
    <%= f.label :author %>
    <%= f.text_field :author %>
  </div>
  <div class="field">
    <%= f.label :email %>
    <%= f.text_field :email %>
  </div>
  <%= render_cell :post, :image_upload_control, f %>
  <%= f.label :content %>
  <%= f.text_area :content %>

  <div class="actions">
    <%= f.submit %>
  </div>
<% end %>

TWO SETS OF _form.html.erb

TWO SETS OF create action

TWO SETS OF strong parameter

TWO SETS OF CSS selector

TWO SETS OF error handling

TWO SETS OF JS DOM manipulation

~~TWO SETS OF tests~~

kekeke

the only differences are

New Topic

will create a topic
and a post
has title field

will create a post
has a topic_id field

○(#‵︿′ㄨ)○ that's it

How to combine them?

Combine them will save brain-power,

making it easier to reuse code,

less potential bugs

We like DRY

清爽

所以每次看到這些重複的時候就十分讓我不爽

We create a dummy model, which wraps Topic and Post

Form Object is good for...

handling multiple models
(treat them as one model)
handling nested models
handling complex form
handling virtual attributes
handling conditional validation
slim your very fat controller logic

4 things a form object has to do

Assign attributes (to inner models)
Read attributes (from inner models)
Validation (of inner models)
Saving (inner models)

Is it easy to make? YES!

Basic Setup

class PostForm

  include ActiveModel::Model

  attr_accessor :topic, :post

Assignment

def attributes(params)
  if new_topic?
    @topic.assign_attributes(topic_params(params))
  end

  @post.assign_attributes(post_params(params))

  @post.images << build_images_from_params(params)
end

Reading

  delegate :title,
           :board_id, to: :topic

  delegate :author, 
           :content,
           :email,
           :topic_id,
           :images, to: :post

Validation

  def valid?
    validity = true
    errors.clear
    [post, topic].each do |object|
      if !object.valid?
        validity = false
        object.errors.each do |key, values|
          errors[key] = values
        end
      end
    end
    validity
  end

Saving to DB

  def save
    return false if !valid?

    ActiveRecord::Base.transaction do
      @post.save! if @post.changed?
      @topic.save! if new_topic?
    end

    true
  rescue ActiveRecord::RecordInvalid => invalid
    false
  end

Result

Both forms are now from the same erb view,

and points to same PostController#create \⊙▽⊙/

There are many ways to build Form Object

and many gems too

reform
virtus

ALL ROADS LEAD TO ROME

Framework's Way can't always help you.

Consider write your own logic to help the framework.

morale of the rewrite #2

Before we go to the next topic...

def show
  @books = [
    "三体",
    "Spin",
    "アイの物語",
  ]
end

<% @books.each do |b| %>
  <%= b %>
<% end %>

def show
end

<% @books.each do |b| %>
  <%= b %>
<% end %>

ERROR

@instance_variable

if you forget to prepare it,
view will error out

From Partials

to Cells Gem

What's Cells

Wraps partial with a mini-controller-action

<%= render partial: "ad",
           locals: {var: var} %>

_ad.html.erb
AD1
<%= var.get_ad() %>

<%= render_cell :ad, :show, var %>

app/cells/ad/show.html.erb
AD1
<%= @var.get_ad() %>

class AdCell < Cell::Rails
  def show(var)
    @var = var
    render
  end
end

準備資料

Cells: benefits

encapsulation

show.html.erb

<% @text = "el psy congroo" %>
<%= render partial: :foo %>

partial

<%= @text %>

show.html.erb

<% @text = "el psy congroo" %>
<%= render_cell :foo, :bar, @text %>

partial

<%= @text %>

No @var contamination

封裝

partial

<% beers = if foo?
  Beer.where("name like ?", b)
elsif bar?
  Beer.where("name like ?", c)
end
%>
<% beers.each do |b| %>
  <%= b.name %>
<% end %>

cell

def show
  @beers = if foo?
    Beer.where("name like ?", b)
  elsif bar?
    Beer.where("name like ?", c)
  end
  render
end

<% @beers.each do |b| %>
  <%= b.name %>
<% end %>

Move model fetching logic out of view

partial

<% cache("#{@foo}-#{@foo.replies.last.updated_at}", expires_in: 60.seconds) do %>
  <%= @foo.title %><br />
  <%= @foo.body %><br />
<% end>

cell

def show(foo)
  @foo = foo
  render
end
cache :show, expires_in: 60.seconds do |cell, foo|
  [@foo, @foo.replies.last.updated_at]
end

cell view

<%= @foo.title %><br />
<%= @foo.body %><br />

Move caching logic out of view

very long cache key logic

helper

module FooHelper
  def foo_body
    # if only used in foo partial
  end
end

cell

def show(foo)
  @foo = foo
  render
end

private

def body
end

cell view

<%= body %>

Move helper closer to its view

partial

<%= foo_body %>

Helper can be local instead of global

For now I recommend cells version 3.11

instead of 4.x

sorry nick don't hit me

Avoid re-rendering

the form if validation error occurs

Last change:

I don't like re-render my form

slow and unresponsive
resets all the Javascript dom-manipulations
if _form.erb is rerendered
in "new" and "create" action
need to ensure @variable are there

@instance_variable

if you forget to prepare it,
view will error out

To re-iterate

  def update
    @book = Book.find(params[:id])

    if @book.update(book_params)
      redirect_to @book, notice: 'Good!'
    else
      render :edit
    end
  end

  def edit
    @book = Book.find(params[:id])
    
    @obscure_var = "52" # which is easy to forget
  end

Ajax comes to rescue!

remote: true

Ajax does not support file upload!

form_for(@post_form) do |f|

form_for(@post_form,
         remote:true,
         format:'json',
         data:{type: :json}) do |f|

remotipart gem

Many iterations until I got out of the pitfall

Form can be submitted via Ajax,

now what?

unobtrusive_flash gem

Passing flash message using cookie
Use JavaScript to display this message

Benefits

One step into cacheable view
(because less personal information on html)
Can display message after
ajax requests
Unify the way message is displayed to the user

Wrap Up

Part III

How is it different to your day to day job?

No chasing after the shiny/latest MySQL/Redis/FooBar
No adding dependency to external packages unless necessary

So I realized I just want an open-source 4chan

Then I found out that something that good exists already
(in PHP)

supports drag/drop upload
instant preview related post
filters 過濾系統
hide posts 隱藏系統
bad post reporting

How do I overcome the sorrow of reinventing the wheel?

$boardQuery = prepare("SELECT COUNT(1) AS 
   'boards_total', SUM(indexed) AS 'boards_public', SUM(posts_total) AS 'posts_total' FROM ``boards``");
$boardQuery->execute() or error(db_error($tagQuery));
$boardResult = $boardQuery->fetchAll(PDO::FETCH_ASSOC)[0];
$boards_hidden  = number_format( $boardResult['boards_total'] - $boardResult['boards_public'], 0 );
$boards_omitted = (int) $searchJson['omitted'];
$posts_total    = number_format( $boardResult['posts_total'], 0 );

I was reminded that why I started learning Rails 3 years ago

Code Mei is a Happy Experience:

1. Ruby aims to make programmers happy

2. Rails framework guides and help me to make decisions

If I am happy,

I guess it is okay to reinvent the wheel <3

Ruby core team

Rails team

and FAQ time

3Q謝謝

thank you

﹨(╯▽╰)∕

Rewriting an Imageboard

By lulalala

Rewriting an Imageboard

5,789

Rewriting an Imageboard

lulalala

Ruby/Rails developer at

When I was a Nooblet

Futaba Channel (2chan)

4chan

Komica

What's so fascinating about it?

So I started Mei

Back and start again

Model

What did I change?

Multi-board support

(Maybe we can call it multi-tenancy)

My Rails imageboard takes around 100+ mb of memory

A PHP imageboard takes around 50+ mb of memory

Steps

Settings

another way to save memory

Put everything under one process

Consider multi-tenancy!

Make Topic & Post look like a single model using Form Object

So there are two forms

but some logics still look VERY similar

Topic

Post

the only differences are

How to combine them?

We like DRY

We create a dummy model, which wraps Topic and Post

Form Object is good for...

4 things a form object has to do

Is it easy to make? YES!

Basic Setup

Assignment

Reading

Validation

Saving to DB

Result

There are many ways to build Form Object

and many gems too

Consider write your own logic to help the framework.

Before we go to the next topic...

@instance_variable

From Partials

to Cells Gem

What's Cells

Cells: benefits

encapsulation

Move model fetching logic out of view

Move caching logic out of view

Move helper closer to its view

Helper can be local instead of global

Avoid re-rendering

the form if validation error occurs

I don't like re-render my form

@instance_variable

Ajax comes to rescue!

Ajax does not support file upload!

Form can be submitted via Ajax,

unobtrusive_flash gem

Benefits

Wrap Up

How is it different to your day to day job?

So I realized I just want an open-source 4chan

Then I found out that something that good exists already (in PHP)

How do I overcome the sorrow of reinventing the wheel?

Code Mei is a Happy Experience:

If I am happy,

I guess it is okay to reinvent the wheel <3

Ruby core team

Rails team

3Q謝謝

Rewriting an Imageboard

More from lulalala

When
I was a Nooblet

Make Topic & Post look like a single model using
Form Object

Then I found out that something that good exists already
(in PHP)