Rebranding
LA Ruby
& Scraping the Meetup API
presented by Anna Mendoza & Josh Loper at Pivotal Labs on 10.09.2014
L.A. Ruby Montly Meetup
why did we
rebrand LA Ruby?
The obvious reasons:
a bit outdated
not much of a resource
barely shared
wasn't being used
The not-so obvious reasons:
MeetUp was becoming the brand
the face of the community isn't represented
is not inviting to new people
and there's a lack of cohesion on the site
FIRST: THE MARK
We started with an exploration
that identified the city with
this group...
Various Applications
THE LANGUAGE
What are we known for:
[ exchange ]
This all informed the
buildout for the website...
SCRAPING THE MEETUP API
First: We had to do some research.
What is the MeetUp API capable of?
The big idea behind this site is to help [ exchange ] information.
We wanted to do a GET request
to get all of the upcoming meetups and
dynamically display them on the site.
How did we do this?
SCRAPING THE MEETUP API
Next: We had to get the JSON
1. We generated an api key from the meetup website:
https://secure.meetup.com/meetup_api/key/
2. Once we had access, we researched the JSON
results and tried to make sense of the data:
JSON RESULTS
SCRAPING THE MEETUP API
3. After getting a handle on the Meetup API, we started
to align the design with technical needs.
Which attributes did we need to save for the database?
rails g model UpcomingMeetup meetup_name:string time:datetime ...
Once the schema was generated, we were ready
to start coding with helper tests, which described what
the scraper method would return.
4. Then we wrote the following methods to scrape the data,
save the list of meetups with a unique meetup_id
( that did not exist in the database - this was important )
and then saved the scraped attributes to a database.
module UpdaterHelper def get_json (source = 'https://api.meetup.com/events&sign=true&photohost=public&group_urlname=laruby&sign=true&key=6667405f41112c7f127c4f243e7243b')...enddef search_meetup_ids(data)... end def save_meetup_ids(hash) ... endend
5. Once we had the meetup data saved to the database,
we started to set up some associations with the
upcoming meetups
EX: Upcoming meetups belonged to a meetup group
and images associated with that group.
5. AND the owners of the group could add
images for the group, and they would be seen
on all upcoming meetups for that group.
Helping both the group to gain an identity
and for the code to be function properly.
Problems we had:
Some of our naming conventions were
different, so we had to rewrite the form
in ActiveAdmin using Formtastic.
We had to scrape organizer names and assign them to
groups in order to associate the meetup with a group.
Problems we had:
We had to set up rails to upload images
to Cloudinary by using a yml configuration
file with the API key in it.
Because Heroku doesn't allow for static images to be uploaded. Thanks heroku!
We wanted to make the json data from the meetup api
match our database as closely as possible;
while still making sense of it for
our application.
Problems we had:
Tracking the urls to each individual
meetup under the dropdown navigation.
How to track each meetup to the urls
in this dropdown became a problem.
First we had to find an upcoming associated with a group,
then we had to search for the id,
and finally present the page.
We did alot of ActiveRecord queries.
Here's a snapshot of the code:
UpcomingMeetups Controller
if params[:id].to_s.length > 8
UpcomingMeetup.select(:id, :group_id).upcoming.distinct.each do |meetup|
if meetup.group.name == params[:id]
@meetup = UpcomingMeetup.where(:id => meetup.id).first
@is_found = true
end
end
if @is_found.nil?
redirect_to ""
end
else
@meetup = UpcomingMeetup.where(:id => params[:id]).first
endend
Gems We Used:
gem 'breadcrumbs_on_rails'
gem 'kaminari'
gem 'cloudinary'
gem 'carrierwave'
gem 'activeadmin', github: 'gregbell/active_admin'
gem 'geocoder'
gem 'devise', '~> 3.3.0'
gem 'rspec-rails', '~> 3.0.0'