com.imperva.stepping.Stepping Maven / Gradle / Ivy

Go to download

Show more of this group Show more artifacts with this name
Show all versions of stepping Show documentation

Stepping is a framework designed to ease the implementation of data processing solutions. In use cases where we need to implement data or data-streaming algorithms or any other processing on data, we need to first handle many different infrastructure issues. For example, we need to decide how to split the data processing logic into different steps, think about our threading policy, how to handle communication between the different steps, error handling etc. One of the most important subjects is the Threading Policy of our solution. For example, we need to think how many threads to open, have the option to distribute the processing of data to multiple 'executors' in parallel, have a thread-safe communication layer between the threads etc. On top of that we also care a lot about the performance of our solution, we want to make sure that the latency added by these infrastructures is minimal as possible. Stepping aims to handle many of these aspects so developers can spend their time on the business logic instead of solving these infrastructure and data flow issues issues over and over again.

There is a newer version: 5.1.0

Show newest version

package com.imperva.stepping;

import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.concurrent.CopyOnWriteArrayList;

public class Stepping {
    private List algos = new CopyOnWriteArrayList<>();
    private HashMap algosMapToControll = new HashMap<>();

    public Stepping register(Algo iAlgo) {
        IAlgoDecorator algo = new AlgoDecorator(iAlgo);
        algos.add(algo);
        return this;
    }

    public Stepping registerAndControl(String id, Algo iAlgo) {
        if (algosMapToControll.containsKey(id))
            throw new SteppingException("Algo id must be unique. Id: " + id + " already exists");

        IAlgoDecorator algo = new AlgoDecorator(iAlgo);
        algosMapToControll.put(id, algo);
        return this;
    }

    public RemoteControllers go() {

        RemoteControllers remoteControllers = new RemoteControllers();
        if (algos.size() > 0) {
            for (Algo algo : algos) {
                algo.init();
            }
        }

        if (algosMapToControll.size() > 0) {
            Iterator> iterator = algosMapToControll.entrySet().iterator();
            while (iterator.hasNext()) {
                Map.Entry pair = iterator.next();
                pair.getValue().init();

                RemoteController remoteController = new RemoteController();
                remoteController.setCloseable(pair.getValue());
                remoteController.setContainer(pair.getValue().getContainer());
                remoteControllers.add(pair.getKey(), remoteController);
            }
        }
        return remoteControllers;
    }
}