Generic Data Conversion Concept

@jurisicmarko

March 2016

Masterdata

hibernate+envers

The Problem

  • Need multiple output representations of persisted objects for different consumers:
    • standard json (cxf+jaxrs automagic) for GUI
    • custom json (only a few fields, some transformations) for PDS
    • h2 database representation for MDI
  • Interface should not change if we rename some fields or do other refactorings (avoid domino-effect spreading to other systems)

Standard approach:

anticorruption/translation layers

  • solves the problem - implementations are independent
  • needs multiple mirrored object hierarchies
  • translators must be updated for most changes (new fields) -> possible source of failure

E. Evans, Domain Driven Design

Our solution

  • Goal h2 database
    • Annotate our objects with new annotations marking what to export to which consumer
    • Use of hibernate dynamic models for H2 export to avoid creating of parallel persistence object hierarchy
    • Convert lsit of objects from our db to list of maps and write it to newly created h2 db
    • Write metadata and pack everything
  • Goal custom JSON consumer
    • Annotate our objects with another annotation
    • Convert list of objects to list of maps
    • let cxf do its magic and produce nice json

Hibernate Dynamic models

  • By default, Hibernate works in normal POJO mode. 
  • Persistent entities don't necessarily have to be represented as POJO classes or as JavaBean objects at runtime.
  • Hibernate also supports dynamic models (using Maps of Maps at runtime)
  • You don't write persistent classes, only mapping files.

Mapping file example

<hibernate-mapping>

    <class entity-name="Customer">

        <id name="id"
            type="long"
            column="ID">
            <generator class="sequence"/>
        </id>

        <property name="name"
            column="NAME"
            type="string"/>

        <property name="address"
            column="ADDRESS"
            type="string"/>

        <many-to-one name="organization"
            column="ORGANIZATION_ID"
            class="Organization"/>

      <!-- ... -->

    </class>
    
</hibernate-mapping>

Writing a map to database

Session s = openSession();
Transaction tx = s.beginTransaction();
Session s = openSession();

// Create a customer
Map david = new HashMap();
david.put("name", "David");

// Create an organization
Map foobar = new HashMap();
foobar.put("name", "Foobar Inc.");

// Link both
david.put("organization", foobar);

// Save both
s.save("Customer", david);
s.save("Organization", foobar);

tx.commit();
s.close();

A few custom annotations, mirroring JPA 

@Target({FIELD, METHOD})
@Retention(RUNTIME)
public @interface MdiColumn {
  String name() default "";

  String type() default "string";

  boolean id() default false;
}
@Target(TYPE)
@Retention(RUNTIME)
public @interface MdiEntity {
  String name() default "";
  String tableName();
}

Entity example

@Entity
@Table(name = "BD_EVU")
@MdiEntity(tableName = "ADMIN_UIC")
public class Evu extends BaseEntity {

  private static final int MIN_LENGTH = 2;
  private static final int MAX_LENGTH = 4;

  @Column
  @NotNull
  @Audited
  @PdsField
  @MdiColumn(id = true, name = "code")
  private Long code;

  @Transient
  @JsonIgnore
  private String description;

  @Column
  @Audited
  @PdsField
  @MdiColumn(name = "name")
  private String name;

  //getters and setters
}

Map method

 @JsonIgnore
  @MdiColumn(name = "privrail_admin_code")
  public Long getPrivRailAdminCode() {
    return privateRailRoute != null ? privateRailRoute.getAdminCode() : null;
  }
  • Useful for some transformations or additional fields

We create the mapping file on the fly

private Configuration getConfiguration(Class modelClass) {
    return new Configuration()
        .setProperty("hibernate.dialect", "org.hibernate.dialect.H2Dialect")
        .setProperty("hibernate.connection.url", "jdbc:h2:file:" + TEMP_DIR + H2_FILENAME + ";MV_STORE=FALSE;MVCC=FALSE")
        .setProperty("hibernate.connection.driver_class", "org.h2.Driver")
        .setProperty("hibernate.connection.username", "sa")
        .setProperty("hibernate.connection.password", "")
        .setProperty("hibernate.default_entity_mode", "dynamic-map")
        .setProperty("hibernate.hbm2ddl.auto", "create")
        .setProperty("hibernate.show_sql", "false")
        .addXML(getHibernateMapping(modelClass));
  }

Save entity list

  private void saveEntities(Class modelClass, Session session, List activeEntities) {
    final Map<String, Method> fieldMappings = getElementMappings(modelClass);
    final String entityName = getEntityName(modelClass);

    Transaction tx = session.beginTransaction();
    activeEntities.forEach(object -> {
      Map<Object, Object> entity = new HashMap<>();
      fieldMappings.entrySet().forEach(entry -> {
        try {
          entity.put(entry.getKey(), entry.getValue().invoke(object));
        } catch (IllegalAccessException | InvocationTargetException e) {
          LOG.error("Error calling getter:" + entry.getValue().getName(), e);
        }
      });
      session.save(entityName, entity);
      exportReferences(session, object);
    });
    tx.commit();
  }

Read annotations and get method mappings

private Map<String, Method> getElementMappings(Class modelClass) {
    Map<String, Method> mappings = new HashMap<>();

    // get regular fields and methods annotated with MdiColumn or MdiCompositeKey
    Stream.concat(
        getAnnotatedElements(modelClass, MdiColumn.class), 
        getAnnotatedElements(modelClass, MdiCompositeKey.class)).forEach(accessibleObject -> {
          // stream only contains MdiColumn or MdiComposityKey ->
          String columnName = accessibleObject.getAnnotation(MdiColumn.class) != null ? 
            accessibleObject.getAnnotation(MdiColumn.class).name() :     
            accessibleObject.getAnnotation(MdiCompositeKey.class).name();
          if (StringUtils.isEmpty(columnName)) {
            columnName = ((Member) accessibleObject).getName();
          }

          if (accessibleObject instanceof Field) {
            try {
              mappings.put(
                columnName, 
                modelClass.getMethod("get" + 
                    StringUtils.capitalize(((Field) accessibleObject).getName())));
            } catch (NoSuchMethodException e) {
              LOG.error("Getter method does not exist", e);
            }
          } else {
            mappings.put(columnName, (Method) accessibleObject);
          }

        });
    return mappings;
  }

Problems

  • Too many annotations on some fields
  • Reflection is slow
  • Mapping and session is created for every entity class (could write for all classes at once but it would make the code more complicated)
  @Column
  @NotNull
  @Size(max = 100)
  @Audited
  @PdsField
  @MdiColumn
  private String name;
private Configuration getConfiguration(Class modelClass) {
    return new Configuration()
  //
        .setProperty("hibernate.hbm2ddl.auto", "create")
  //
  }

References

DEMO

Generic Data Conversion

By Marko Jurišić

Generic Data Conversion

  • 826