Spring Data + MongoDB

Nikos Kitmeridis

Software Developer @ AMD Telecom 

https://gr.linkedin.com/in/nikos-kitmeridis-8b17b23a

Overview of Spring Data

POJOs              Various forms of storage

Sub-projects exist for different backend technologies

•  SQL
  • JDBC, JPA...
• NoSQL
  • MongoDB, Cassandra, Redis...
• Other
  • Apache SOLR, APACHE Hadoop, REST...

Spring Data Basics

• Repository
  • Convert retrieved data into POJOs
  • Convert POJOs into saved data
• Mapping
  • Inferred from conventions
  • Annotations to help automate field mapping
• Template
  • Provide simplified direct-access to database
    • Automatically managing internal resources
• Query
  • Apply provided native queries directly
  • Convert QueryDSL into native queries

Spring data MongoDB - Quick start

Define your model

• Simple POJO
  • private fields of the POJO are mapped to document field in your MongoDb collection
• Spring data annotations defining meta-data about the collection and documents taht will be created in MongoDb
  • @Document
  • @TypeAlias
  • @CompoundIndexes
    • @CompoundIndex
  • @Id
  • @Indexed
  • @Transient
  • ...

Spring data MongoDB - Quick start

Define your model

Example

@Document(collection = "account")
@TypeAlias("Account")
@CompoundIndexes({
    @CompoundIndex(name = "company_info", def = "{'industry': 1, 'type': 1}")
})
public class Account {

    @Id
    private String id;

    @Indexed(name = "organizationName_1", unique = true)
    private String organizationName;

    private String industry;

    private OrganizationType type;

    private Long noOfUsers;

    @Transient
    private Set<User> users;

    @Indexed(name = "createdAt_1")
    private Date createdAt;

...
}

Spring data MongoDB - Quick start

• Declare your Repository extending a predefined Spring Data Repository. 

// Central repository marker interface.
public interface Repository<T, ID> {}

// Interface for generic CRUD operations on a repository for a specific type.
public interface CrudRepository<T, ID extends Serializable> extends Repository<T, ID> {

	<S extends T> S save(S entity);

	<S extends T> Iterable<S> save(Iterable<S> entities);

	T findOne(ID id);

	boolean exists(ID id);

	Iterable<T> findAll();

	Iterable<T> findAll(Iterable<ID> ids);

	long count();

	void delete(ID id);

	void delete(T entity);

	void delete(Iterable<? extends T> entities);

	void deleteAll();
}

// Extension of CrudRepository to provide additional methods to retrieve entities using the pagination and
public interface PagingAndSortingRepository<T, ID extends Serializable> extends CrudRepository<T, ID> {

	Iterable<T> findAll(Sort sort);

	Page<T> findAll(Pageable pageable);
}

Defining Finders

• Auto-generated finders obey naming conventions
  • findBy<DataMember><Op> 
  • <Op> can be Gt, Lt, Ne, Between, Like ... etc
• Alternatively, '@Query' annotation takes as parameter JSON formed mongodb queries (no conventions in the method's name)

public interface UserRepository extends PagingAndSortingRepository<User, String> {
    
    User findByUsername(String Username); // No <Op> for Equals

    User findByLastnameOrderByFirstname(String lastname);

    User findByBirthdateBetween(Date date1, Date date2);

    @Query(value = "{'birthdate': {$gte: ?0, $lte: ?1}}")
    Collection<Account> findByBirthDateBetween(Date date1, Date date2);
    
}

Implementing Custom Repositories

3 Steps to implement Custom Repositories

• Declare the custom repository interface

interface UserRepositoryCustom {
  public void someCustomMethod(User user);
}

• Default repository must extend your custom repositoy

interface UserRepository extends CrudRepository<User, Long>, UserRepositoryCustom {

  // Declare query methods here
}

• Create implementation class of your custom repository (naming convention: <DefaultRepositoryName>Impl)

class UserRepositoryImpl implements UserRepositoryCustom {

  public void someCustomMethod(User user) {
    // Your custom implementation
  }
}

MongoTemplate

The MongoTemplate class  is the central class of the Spring’s MongoDB support providing a rich feature set to interact with the database.

• implements the interface MongoOperations
• MongoTemplate  VS MongoDB Driver:
  • domain objects instead of DBObject 
  • fluent APIs (DSL) for Query, Criteria, and Update operations instead of populating a DBObject to specify the parameters for those operations.

MongoOperations

public interface MongoOperations {

	String getCollectionName(Class<?> entityClass);
	...
	CommandResult executeCommand(String jsonCommand);
	...
	void executeQuery(Query query, String collectionName, DocumentCallbackHandler dch);
	...
	<T> DBCollection createCollection(Class<T> entityClass);
	...
	<T> GroupByResults<T> group(Criteria criteria, String inputCollectionName, GroupBy groupBy, Class<T> entityClass);
	...
	<O> AggregationResults<O> aggregate(TypedAggregation<?> aggregation, String collectionName, Class<O> outputType);
	...
	<T> MapReduceResults<T> mapReduce(String inputCollectionName, String mapFunction, String reduceFunction, Class<T> entityClass);
	...
	<T> List<T> find(Query query, Class<T> entityClass);
	...
	<T> T findAndModify(Query query, Update update, Class<T> entityClass);
	...
	void insert(Object objectToSave, String collectionName);
	...
	WriteResult upsert(Query query, Update update, Class<?> entityClass, String collectionName);
	...
	WriteResult updateFirst(Query query, Update update, Class<?> entityClass);
	...
	WriteResult updateMulti(Query query, Update update, Class<?> entityClass);
	...
	WriteResult remove(Object object);

	...

}

Spring Data MongoDB + Unit Testing

NoSQLUnit

• NoSQLUnit is an open source JUnit extension for writing tests of Java applications that use NoSQL databases.
• The goal of NoSQLUnit is to manage the lifecycle of NoSQL engines.
  • maintain the databases under test into known state
  • standardize the way we write tests for NoSQL applications.
• Supports the following engines:
  • MongoDB
  •  Cassandra
  •  HBase
  •  Redis
  •  Neo4j.

NoSQLUnit MongoDB

It's all about JSON Datasets

Initial Dataset

Expected DataSet

"user": [
 {
   "_id": "u1",
   "userName": "allan123",
   "firstName": "Allan",
   "lastName": "Connolly",
   "email": "OLDEMAIL@example.com"
 }
]

"user": [
 {
   "_id": "u1",
   "userName": "allan123",
   "firstName": "Allan",
   "lastName": "Connolly",
   "email": "NEWEMAIL@example1.com"
 }
]

Unit Test's executed query

db.user.update(
    { "_id": "u1" },
    { $set: { "email": "NEWEMAIL@example.com" } },
    { upsert: true }
)

NoSQLUnit MongoDB

Seeding Database

@UsingDataSet

• Class or Method level annotation
  •  If there is definition on both, test level annotation takes precedence.
• Properties
  • locations
    • Points to JSON file(s) containing the initial dataset needed to be stored in the DB before running the test
    • Locations are relative to test class location.
  • ​loadStrategy
    • ​INSERT: Insert defined datasets before executing any test method.
    • DELETE_ALL: Deletes all elements of database before executing any test method.
    • CLEAN_INSERT (default): It deletes all elements of database and then insert defined datasets before executing any test method.

NoSQLUnit MongoDB

Verifying Database

@ShouldMatchDataSet

• Optional annotation
  • By using @ShouldMatchDataSet on test method, NoSQLUnit will check if database contains expected entries after test execution.
    • ​asserting database state directly from testing code (may imply a huge amount of work)

NoSQLUnit MongoDB

Avoiding specific field check

@IgnorePropertyValue

• Optional annotation
  • Sometimes tests produce field values at runtime that is not easy to preassume
    • eg: Date updatedAt = new Date(System.currentMillis());
• Used with
  • @CustomComparisonStrategy
  • @ShouldMatchDataSet​​
• ​​Properties
  • properties 
    • collection.property :  the exclusion only will affect to the indicated collection.
    • property : the exclusion will affect all the collections of the DB.

NoSQLUnit MongoDB

Example

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(classes = TestConfiguration.class)
@CustomComparisonStrategy(comparisonStrategy = MongoFlexibleComparisonStrategy.class)
public class AccountRepositoryTester {

    @Rule
    public MongoDbRule mongoDbRule = newMongoDbRule().defaultSpringMongoDb("springdata-poc-test");

    @Autowired
    private ApplicationContext applicationContext;

    @Autowired
    private AccountRepository accountRepository;
  
    @Test
    @UsingDataSet(locations = "Initial-AccountDataSet.json")
    @ShouldMatchDataSet(location = "Expected-AccountDataSet.json")
    @IgnorePropertyValue(properties = {"account.updatedAt"}) 
    public void test_UpdateAccountsField_whenAccountExistsAndFieldExists_updatesAccount() {

        accountRepository.updateAccountsField("c1", "industry", "Financial");
    }
}

Thank You!!!

Useful Links

Introduction to Spring Data JPA and Spring Data MongoDB

(by Nik Trevallyn Jones, many slides of the current presentation are based on this video)

https://www.youtube.com/watch?v=P05GlyrIz0o

__________________________________________________________

Spring Data MongoDb Documentation

http://docs.spring.io/spring-data/mongodb/docs/current/reference/html/

__________________________________________________________

NoSQLUnit Source code & Documentation

https://github.com/lordofthejars/nosql-unit

___________________________________________________________________________________

Simple POC on Spring data Mongodb & NoSQLUnit

https://bitbucket.org/nkitmeri/springdata-mongodb-poc