infinispan-queryable-custom-loader

This repository contains step by step instructions of how to get a custom loader working with querying. Each step is accompanied by a related commit that shows progress.

Note	This project is not meant to be checked out, but to create a project for yourself and can follow step by step

Steps

1. (Optional) Configure a maven-settings.xml to point to a local repository if needed. The repository comes with an empty one (example-maven-settings.xml) that can be used by replacing the directory location of the location on the filesystem. All subsequent commands will assume this is done, you can remove all subsequent "-s maven-settings.xml" references if this was not done.

2. Generate the store archetypes (substitute the version as appropriate). Note the verison is not the same as DataGrid, but rather specific to the archetype. As of this writing the current version is 1.0.25.

   mvn -s maven-settings.xml archetype:generate \
        -DarchetypeGroupId=org.infinispan.archetypes \
        -DarchetypeArtifactId=store-archetype  \
        -DarchetypeVersion=<version>

   This command will eventually prompt the user for a groupId, artifactId, project version and package. Please populate as appropriate for the usage. This example will use test, cacheloader, 1.0-SNAPSHOT and test.cacheloader.impl respectively.

   You should now have a cacheloader directory in the current directory that contains a pom.xml and several children modules, one per type of custom loader/store. To simplify the project all the contents of cacheloader will be moved to the root directory. In this example we will be using the cache-loader project moving forward.

   Note	You may notice that the generated projects do not compile, we will be fixing this in the next step

3. Adding DataGrid dependencies to the base project

   1. Configure Infinispan bom to parent pom

      Add the following snippet to the pom.xml in the root directory replacing 9.4.11.Final with the appropriate version

         <parent>
            <groupId>org.infinispan</groupId>
            <artifactId>infinispan-bom</artifactId>
            <version>9.4.11.Final</version>
         </parent>

   2. Replace Infinispan embedded dependency with core and add protostream dependency

            <dependency>
               <groupId>org.infinispan</groupId>
               <artifactId>infinispan-core</artifactId>
               <scope>provided</scope>
            </dependency>
      
            <dependency>
               <groupId>org.infinispan.protostream</groupId>
               <artifactId>protostream</artifactId>
               <scope>provided</scope>
            </dependency>

      Note	The dependency is marked as provided scope, since the Infinispan server will provide those classes automatically and aren’t needed in a built jar.

   3. (Optional) Remplace metainf version to use bom. Remove version.metainf property.

            <dependency>
               <groupId>org.kohsuke.metainf-services</groupId>
               <artifactId>metainf-services</artifactId>
               <version>${version.metainf-services}</version>
               <scope>provided</scope>
            </dependency>

      Note	The dependency is marked as provided scope, this is due to this dependency only being required for compile and is not used at runtime

4. Setup configuration classes, allows for parameters via xml

   1. We start by removing some redundant classes. These are only useful in embedded mode, which we are using server

      1. Attribute.java

      2. CustomStoreConfigurationParser.java

      3. Element.java

   2. Now the various configuration parameters need to be added to the following classes

      Here is an example of adding a name and age to our configuration

      1. CustomStoreConfiguration.java

         @BuiltBy(CustomStoreConfigurationBuilder.class)
         @ConfigurationFor(CustomCacheLoader.class)
         public class CustomStoreConfiguration extends AbstractStoreConfiguration {
         
             static final AttributeDefinition<String> NAME = AttributeDefinition.builder("name", null, String.class).immutable().build();
             static final AttributeDefinition<Integer> AGE = AttributeDefinition.builder("age", 0, Integer.class).immutable().build();
         
             private final Attribute<String> name;
             private final Attribute<Integer> age;
         
             public static AttributeSet attributeDefinitionSet() {
                 return new AttributeSet(CustomStoreConfiguration.class, AbstractStoreConfiguration.attributeDefinitionSet(), NAME, AGE);
             }
         
             public CustomStoreConfiguration(AttributeSet attributes, AsyncStoreConfiguration async, SingletonStoreConfiguration singletonStore) {
                 super(attributes, async, singletonStore);
                 name = attributes.attribute(NAME);
                 age = attributes.attribute(AGE);
             }
         
             public String name() {
                 return name.get();
             }
         
             public Integer age() {
                 return age.get();
             }
         }

      2. CustomStoreConfigurationBuilder.java

         public class CustomStoreConfigurationBuilder extends AbstractStoreConfigurationBuilder<CustomStoreConfiguration, CustomStoreConfigurationBuilder>{
         
             public CustomStoreConfigurationBuilder(
                   PersistenceConfigurationBuilder builder) {
                 super(builder, CustomStoreConfiguration.attributeDefinitionSet());
             }
         
             public CustomStoreConfigurationBuilder name(String name) {
                 attributes.attribute(NAME).set(name);
                 return this;
             }
         
             public CustomStoreConfigurationBuilder age(int age) {
                 attributes.attribute(AGE).set(age);
                 return this;
             }
         
             @Override
             public CustomStoreConfiguration create() {
                 return new CustomStoreConfiguration(attributes.protect(), async.create(), singletonStore.create());
             }
         
             @Override
             public CustomStoreConfigurationBuilder self() {
                 return this;
             }
         }

   3. The CustomCacheLoader can inject the configuration in the init method

      This can then be used in an invocation later

          CustomStoreConfiguration config;
      
          @Override
          public void init(InitializationContext ctx) {
              config = ctx.getConfiguration();
          }

5. Add user classes required for retrieving from the custom store and handle serialization

   1. Change the pom.xml to include user dependencies in resulting jar

      This also includes manifest entries to expose the required modules to the loader (ie. core and protostream)

               <!-- This plugin will pull all the non provided scoped dependencies into a resulting jar, allowing for
                    all the classes to be available to the deployment -->
               <plugin>
                  <artifactId>maven-assembly-plugin</artifactId>
                  <executions>
                     <execution>
                        <phase>package</phase>
                        <goals>
                           <goal>single</goal>
                        </goals>
                     </execution>
                  </executions>
                  <configuration>
                     <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                     </descriptorRefs>
                     <archive>
                        <manifestEntries>
                           <Dependencies>org.infinispan.core:jdg-7.3 services, org.infinispan.protostream:jdg-7.3 services</Dependencies>
                        </manifestEntries>
                     </archive>
                  </configuration>
               </plugin>

      This will create a new jar in the target directory named as cache-loader-1.0-SNAPSHOT-jar-with-dependencies.jar which will contain all dependencies that are scoped as compile (default scope).

   2. Add dependencies of user actual classes

      This can be done a couple different ways, however the first method is the recommended way to prevent duplication of code across projects.

      1. The preferred method of adding user classes is just to add them as a dependency in the pom.xml directly

         The following is just an example and should be replaced with user specific modules and versions

               <dependency>
                  <groupId>some.organization</groupId>
                  <artifactId>domain-objects</artifactId>
                  <version>${version.some.organization}</version>
               </dependency>
               <dependency>
                  <groupId>some.organization</groupId>
                  <artifactId>domain-access-objects</artifactId>
                  <version>${version.some.organization}</version>
               </dependency>

         These dependencies its transitive dependencies will automatically be added to the resulting with-dependencies jar.

      2. It is also possible to add the .java files directly to the project

         This project will use this approach since it is simpler as an example and shows the actual user classes

         This is a contrived example just to show interactions with the loader.

         public class Person {
         
             private final long id;
             private final String name;
             private final Integer age;
         
             public Person(long id, String name, Integer age) {
                 this.id = id;
                 this.name = name;
                 this.age = age;
             }
         
             public long getId() {
                 return id;
             }
         
             public String getName() {
                 return name;
             }
         
             public Integer getAge() {
                 return age;
             }
         }

6. Initialize the instance variables in the cache loader

   The cache loader methods will need some shared variables to work properly. These include things such as the marshalledEntryFactory, byteBufferFactory and configuration of the loader as done before.

   Note	Only initializing needed objects from the InitializationContext should be done in the init method

       ByteBufferFactory byteBufferFactory;
       MarshalledEntryFactory marshalledEntryFactory;
       CustomStoreConfiguration config;
   
       @Override
       public void init(InitializationContext ctx) {
           config = ctx.getConfiguration();
           byteBufferFactory = ctx.getByteBufferFactory();
           marshalledEntryFactory = ctx.getMarshalledEntryFactory();
       }

7. (Optional) Configuring Protostream marshalling

   This is needed if the entries loaded from the cache loader can be queryable

   1. We start by defining the protobuf schema for the class(es) that will be stored in the Data Grid

      person.proto

      package test;
      
      message Person {
        required int64 id = 1;
        optional string name = 2;
        optional int32 age = 3;
      }

      Make sure to note the package and message name (ie. test.Person) this is used later

   2. Next we need to write the actual marshaller for the class

      PersonMarshaller.java

      public class PersonMarshaller implements MessageMarshaller<Person> {
         @Override
         public Person readFrom(ProtoStreamReader reader) throws IOException {
            Long id = reader.readLong("id");
            String name = reader.readString("name");
            Integer age = reader.readInt("age");
            return new Person(id, name, age);
         }
      
         @Override
         public void writeTo(ProtoStreamWriter writer, Person person) throws IOException {
            writer.writeLong("id", person.getId());
            writer.writeString("name", person.getName());
            writer.writeLong("age", person.getAge());
         }
      
         @Override
         public Class<? extends Person> getJavaClass() {
            return Person.class;
         }
      
         @Override
         public String getTypeName() {
            return "test.Person";
         }
      }

      The getTypeName method must return the String value from the protobuf schema (ie. <package name>.<message name>).

   3. Create the protostream serialization context on startup

      With the schema and marshaller we can now setup the serialization context that will be used to convert the user object to the proper storage format.

      CustomCacheLoader.java

          SerializationContext ctx;
      
          private static final String PROTOBUF_DEFINITION_RESOURCE = "person.proto";
      
          @Override
          public void start() {
              ctx = ProtobufUtil.newSerializationContext();
      
              try {
                  ctx.registerProtoFiles(FileDescriptorSource.fromResources(CustomCacheLoader.class.getClassLoader(),
                        PROTOBUF_DEFINITION_RESOURCE));
              } catch (IOException e) {
                  throw new CacheException(e);
              }
      
              ctx.registerMarshaller(new PersonMarshaller());
          }
      
          @Override
          public void stop() {
              ctx = null;
          }

      Note	We have to pass the ClassLoader of the CustomCacheLoader so it can access all the classes in the custom loader jar.

8. Implement the actual CacheLoader methods

   In this step we will just be implementing the load method as an example. The concepts of it can be applied to all the other methods though.

   CustomCacheLoader.java

       @Override
       public MarshalledEntry<K, V> load(Object key) {
           byte[] keyBytes = ((WrappedByteArray) key).getBytes();
           // The key will be in serialized form, so we need to deserialize it to store in the Person
           Long id;
           try {
               id = ProtobufUtil.fromWrappedByteArray(ctx, keyBytes);
           } catch (IOException e) {
               throw new CacheException(e);
           }
           // This would be where the code to talk to a database etc to retrieve the user object instance
           Person loadedPerson = new Person(id, config.name(), config.age());
   
           byte[] valueBytes;
           try {
               valueBytes = ProtobufUtil.toWrappedByteArray(ctx, loadedPerson);
           } catch (IOException e) {
               throw new CacheException(e);
           }
           return marshalledEntryFactory.newMarshalledEntry(keyBytes, valueBytes, null);
       }

   With that done, the loader should be ready to go!

9. Deploy and configure the loader in the Server

   Note	This section is not included in the commit since it is done in an external system

   1. Copy the cache-loader-1.0-SNAPSHOT-jar-with-dependencies.jar file to the Data Grid standalone/deployments folder

   2. Change the standalone.xml/clustered.xml to use the loader

      clustered.xml

      <distributed-cache name="default">
        <encoding>
          <key media-type="application/x-protostream"/>
          <value media-type="application/x-protostream"/>
        </encoding>
        <persistence>
          <store class="test.cacheloader.impl.CustomCacheLoader">
            <property name="name">Joe</property>
            <property name="age">32</property>
          </store>
        </persistence>
      </distributed-cache>

      This assumes the client is using ProtoStreamMarshaller and sets the encoding type of store objects as protostream. This is useful as we can convert between this storage type and others quite easily.

10. (Testing) Access the data from REST end point

    Now that we have the loader present and have the media type set we can upload the proto schema to the server and access the data idirectly via REST (Requires user realm authentication)

    1. Upload proto schema via REST

       curl -u <user>:<pass> -X POST --data-binary @./person.proto http://127.0.0.1:8080/rest/___protobuf_metadata/person.proto

    2. Invoke a REST end point to retrieve the user data

       We have to specify the key content type as String won’t work

       curl -u <user>:<pass> --header "Key-Content-Type: application/x-java-object;type=java.lang.Long" http://127.0.0.1:8080/rest/default/3
       
       {
          "_type": "test.Person",
          "id": "3",
          "name": "Joe",
          "age": 32
       }