• Home
• com.netflix.eureka2
• eureka-core
• 2.0.0-DP2
• source code
• Index.java

×

Project download

Many resources are needed to download a project. Please understand that we have to compensate our server costs. Thank you in advance.
Project price only 1 $

You can buy this project and download/modify it how often you want.

com.netflix.eureka2.interests.Index Maven / Gradle / Ivy

package com.netflix.eureka2.interests;

import com.netflix.eureka2.datastore.NotificationsSubject;
import rx.Observable;
import rx.Subscriber;
import rx.functions.Func1;
import rx.observables.ConnectableObservable;
import rx.subjects.PublishSubject;
import rx.subjects.Subject;

import java.util.Iterator;


/**
 * An index implementation associated with an {@link Interest}. 

 * An index contains two primary sources of data:
 * 

 
Initial data
 
Real time data
 
 *
 * The "real time data" is piped from the source of data for this index and the "initial data" is an optional set of
 * data that is sent to any {@link Subscriber} of this index before any {@link ChangeNotification} is sent.
 *
 * If the initial data source (implemented as {@link Index.InitStateHolder}) is empty, then it is assumed that the real
 * time data source replays all initial data which will be required for any subscriber of this interest to create the
 * complete view of the data.
 *
 * 
Why do we need two sources?
 *
 * Typically an index is used to create streams of data for a matching {@link Interest}. Since, an index here is only
 * publishing a {@link ChangeNotification} it is imperative that all the notifications from the start of time are sent
 * to the {@link Subscriber} so that it can create a full view of the data that it is interested in.
 *
 * Now, in order to relay all {@link ChangeNotification}'s to all subscribers, we need to maintain all these
 * notifications for the entire lifetime of a server. This can be quiet expensive if the original data-source changes
 * often.
 *
 * So, it seems quiet obvious that we need to compact these notifications when possible. eg: If an item is deleted, any
 * new subscriber do not need to know about it at all. So, this notification can be ignored by any new subscriber that
 * subscribed after the item is deleted.
 *
 * It is very important for the sanity of this data that all the {@link ChangeNotification} from a single source are
 * completely ordered. So, it is also obvious that we need to pass this data through a queue. However, the presence of
 * a queue creates an unnecessary queuing point for subscribers who have already got the initial state as the order is
 * maintained by the source.
 *
 * For this reason, we have two data sources, and any subscriber to this index receives notifications from these sources
 * in order. In all cases, all notifications from the initial data source is sent before the real time data is sent to
 * the subscriber.
 *
 * 
Is there any message loss between two sources?
 *
 * All {@link ChangeNotification}s received by this index which are essentially the notifications from the original
 * data source applicable to this index (i.e. {@link Interest#matches(Object)} returns {@code true}) are sent to both
 * these data sources (initial and real time).
 * At the start of a subscription the {@link Index.InitStateHolder} makes sure that the returned {@link Iterator} has
 * all data that is received till now and no more data is added to the real time source till this iterator is created.
 *
 * 
How does this guarantee ordering between sources?
 *
 * We cache all change notifications from the real time data source, till all the notifications from the initial data
 * source is sent to the subscriber. Hence, the change notifications from the real time data source never reaches the
 * subscriber till the init state is completed.
 *
 * @author Nitesh Kant
 */
public class Index extends Subject, ChangeNotification> {

    private final Interest interest;
    private final NotificationsSubject notificationsSubject;

    protected Index(final Interest interest, final InitStateHolder initStateHolder,
                    final PublishSubject> realTimeSource) {
        super(new OnSubscribe>() {
            @Override
            public void call(Subscriber> subscriber) {
                ConnectableObservable> realTimePublish = realTimeSource.publish();
                realTimePublish.subscribe(subscriber); // For real time notifications.
                for (ChangeNotification notification : initStateHolder) {
                    subscriber.onNext(notification);
                }
                realTimePublish.connect(); // This makes sure that there is complete order between init state & real time.
            }
        });
        this.interest = interest;
        this.notificationsSubject = initStateHolder.getNotificationSubject();
        this.notificationsSubject.subscribe(initStateHolder);// It is important to ALWAYS update init state first otherwise, we will lose data (see class javadoc)
        this.notificationsSubject.subscribe(realTimeSource);
    }

    public Interest getInterest() {
        return interest;
    }

    @Override
    public void onCompleted() {
        notificationsSubject.onCompleted();
    }

    @Override
    public void onError(Throwable e) {
        notificationsSubject.onError(e);
    }

    @Override
    public void onNext(ChangeNotification notification) {
        notificationsSubject.onNext(notification);
    }

    public static  Index forInterest(final Interest interest,
                                           final Observable> dataSource,
                                           final InitStateHolder initStateHolder) {
        PublishSubject> realTimeSource = PublishSubject.create();
        Index toReturn = new Index(interest, initStateHolder, realTimeSource);

        dataSource.filter(new Func1, Boolean>() {
            @Override
            public Boolean call(ChangeNotification notification) {
                return interest.matches(notification.getData());
            }
        }).subscribe(toReturn); // data source sends all notifications irrespective of the interest set. Here we filter based on interest.
        return toReturn;
    }

    @Override
    public boolean hasObservers() {
        return notificationsSubject.hasObservers();
    }

    /**
     * An initial data source for {@link com.netflix.eureka2.interests.Index}.
     *
     * 
Producer
     * There will always be a single producer (even if multiple the updates will be sequenced out of this context of
     * this holder) queue.
     *
     * The producer will always be the {@link com.netflix.eureka2.interests.Index}, the updates to which
     * (onNext/onError/onComplete) are always sequence i.e. it is not invoked concurrently.
     *
     * 
Consumers
     * There will be multiple consumers of this data i.e. multiple consumers can call {@link #iterator()} on this class.
     *
     * 
Consistency guarantees
     *
     * This implementation should always have a consistency between the data written by
     * {@link #onNext(ChangeNotification)} and what is returned in the {@link #iterator()}. There should not be any
     * missed notifications even if {@link #onNext(ChangeNotification)} and {@link #iterator()} are called concurrently.
     *
     * @param  Type of data that this holds.
     */
    protected static abstract class InitStateHolder extends Subscriber>
            implements Iterable> {

        protected final Iterator> EMPTY_ITERATOR = new Iterator>() {
            @Override
            public boolean hasNext() {
                return false;
            }

            @Override
            public ChangeNotification next() {
                return null;
            }

            @Override
            public void remove() {
            }
        };

        private volatile boolean done;
        private final NotificationsSubject notificationSubject;

        protected InitStateHolder(NotificationsSubject notificationSubject) {
            this.notificationSubject = notificationSubject;
        }

        @Override
        public Iterator> iterator() {

            if (isDone()) {
                return EMPTY_ITERATOR;
            }

            try {
                notificationSubject.pause();
                return _newIterator();
            } finally {
                notificationSubject.resume();
            }
        }

        @Override
        public final void onCompleted() {
            done = true;
            clearAllNotifications(); // Completion == shutdown, so after this, there isn't anything to be done.
        }

        @Override
        public final void onError(Throwable e) {
            done = true; // Since, any one interested in this source will also be interested in the real time source,
                         // we leave it to the real time source to propagate this error. We just return an empty iterator
                         // whenever the upstream source is done (i.e. onComplete/onError on this instance)
            clearAllNotifications(); // Completion == shutdown, so after this, there isn't anything to be done.
        }

        @Override
        public final void onNext(ChangeNotification notification) {
            // Since we pause notifications during iterator creation, we will not get an onNext when iterator creation is in progress.
            addNotification(notification);
        }

        protected NotificationsSubject getNotificationSubject() {
            return notificationSubject;
        }

        protected boolean isDone() {
            return done;
        }

        protected abstract void addNotification(ChangeNotification notification);

        protected abstract void clearAllNotifications();

        protected abstract Iterator> _newIterator();
    }
}