Leveraging Opscenter to make Cassandra Easy

Full disclosure about me

  • I work for Datastax, but this is not an official Datastax talk
  • I've talked about tons of different frameworks and libraries over the years without being a sales pitch
  • I do think opscenter is pretty awesome

What is Opscenter?

  • Provisioning tool
  • Monitoring and Alerting tool
  • Simplifies many cassandra operations
  • Works with both DSE and DSC (although some features are DSE only)
  • Swiss army knife for Ops teams

How to get started?

  • Part of datastax Sandbox http://www.datastax.com/download#dl-sandbox
  • Tarball install http://www.datastax.com/download#dl-opscenter
  • Package install (yum/apt-get)
  • Once running goto http://localhost:8888/ (or whatever your hostname is)

Manage Existing Cluster

  • Possible to provision new as well, but most people trying this will probably have a cluster
  • Pick a cassandra node to connect to (same criteria as a seed node)
  • If using non-standard rpc/thrift ports, specify them.

 

Depending on your cluster configuration, you may need to extend the timeout:

[clusters]
add_cluster_timeout = 30

Agents?

  • Right now we have basic ring topology
  • To get detailed information we need to install agents
  • The Fix link at the top of the screen will install agents for us
  • Agents have a small memory and cpu footprint, they are installed on each cassandra node and communicate with that node via JMX
  • Much of the heavy lifting is pushed to the Opscenter server

The High Level

Trouble shooting Installation

  • Opscenter to cassandra? cqlsh host port
  • Opscenter to agent? curl -i host:61621/alive
  • Agent to cassandra? jconsole/jmxterm/etc
  • Agent/Opsc Stomp? (check logs)
  • Set logging to debug http://www.datastax.com/documentation/opscenter/5.0/opsc/configure/opscConfigLogging.html
  • Check permissions
  • Bounce Opscenterd

Node Operations

  • These are the kinds of operations you can also accomplish using nodetool
  • Select a node from the ring view to bring up details, then issue an action
  • In DSE it is possible to have a bulk action (although we document our API and you could script your own)
  • (api docs https://www.datastax.com/documentation/opscenter/5.0/api/docs/index.html)

Cluster Operations

  • Cassandra Config!
  • Add nodes
  • Restart
  • Rebalance
  • https://gist.github.com/philipsdoctor/5a34dfcfd1fe1f5c6d39

Monitoring

  • Create custom dashboards
  • Add graphs
  • DSE allows user specific graphs

DSE Only

  • Best Practice Services
  • Backup/Restore
  • Repair 
  • Repair reading 
  • http://www.datastax.com/documentation/opscenter/5.0/opsc/online_help/services/repairServiceErrors.html
  • http://www.datastax.com/documentation/opscenter/5.0/opsc/online_help/services/repairServiceHowItWorks.html

Reading

  • Opscenter docs http://www.datastax.com/documentation/opscenter/5.0/opsc/about_c.html
  • Ports http://www.datastax.com/documentation/opscenter/5.0/opsc/reference/opscPorts_r.html
  • Advanced Config
  • http://www.datastax.com/documentation/opscenter/5.0/opsc/configure/opscConfigProps_r.html 
Made with Slides.com