io.nosqlbench.engine.api.activityconfig.yaml.OpTemplate Maven / Gradle / Ivy
Go to download
Show more of this group Show more artifacts with this name
Show all versions of adapters-api Show documentation
Show all versions of adapters-api Show documentation
The DriverAdapter API for nosqlbench;
Provides the interfaces needed to build drivers that can be loaded
by nosqlbench core, using a minimal and direct API for op mapping.
/*
* Copyright (c) 2022 nosqlbench
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.nosqlbench.engine.api.activityconfig.yaml;
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import io.nosqlbench.api.engine.util.Tagged;
import io.nosqlbench.api.config.params.Element;
import io.nosqlbench.api.config.params.NBParams;
import io.nosqlbench.api.config.standard.NBTypeConverter;
import io.nosqlbench.api.errors.OpConfigError;
import io.nosqlbench.virtdata.core.templates.ParsedStringTemplate;
import java.util.LinkedHashMap;
import java.util.Map;
import java.util.Objects;
import java.util.Optional;
import java.util.function.Function;
/**
* The OpTemplate is the developer's view of the operational templates that users
* provide in YAML or some other structured format.
*
* Terms
* Within this documentation, the word OpTemplate will refer to the template API and
* semantics. The word user template will refer to the configuration data as provided
* by a user.
*
* OpTemplates are the native Java representation of the user templates that specify how to
* make an executable operation. OpTemplates are not created for each operation, but are used
* to create an mostly-baked intermediate form commonly known as a ready op.
* It is the intermediate form which is used to create an instance of an executable
* op in whichever way is the most appropriate and efficient for a given driver.
*
* This class serves as the canonical documentation and API for how user templates
* are mapped into a fully resolved OpTemplate. User-provided op templates can be
* any basic data structure, and are often provided as part of a YAML workload file.
* The description below will focus on structural rules rather than any particular
* encoding format. The types used are fairly universal and easy to map from one
* format to another.
*
*
* A long-form introduction to this format is included in the main NoSQLBench docs
* at docs.nosqlbench.io
* under the Designing Workloads section.
*
* A few structural variations are allowed -- No specific form enforced. The reasons for this are:
* 1) It is generally obvious what as user wants to do from a given layout. 2) Data structure
* markup is generally frustrating and difficult to troubleshoot. 3) The conceptual domain of
* NB op construction is well-defined enough to avoid ambiguity.
*
* Type Conventions
*
* For the purposes of simple interoperability, the types used at this interface boundary should
* be limited to common scalar types -- numbers and strings, and simple structures like maps and lists.
* The basic types defined for ECMAScript should eventually be supported, but no domain-specific
* objects which would require special encoding or decoding rules should be used.
*
* Standard Properties
*
* Each op template can have these standard properties:
*
* - name - every op template has a name, even if it is auto generated for you. This is used to
* name errors in the log, to name metrics in telemetry, and so on.
* - description - an optional description, defaulted to "".
* - statement - An optional string value which represents an opaque form of the body of
* an op template
* - params - A string-object map of zero or more named parameters, where the key is taken as the parameter
* name and the value is any simple object form as limited by type conventions above.
*
- bindings - A map of binding definitions, where the string key is taken as the anchor name, and the
* string value is taken as the binding recipe.
* - tags - A map of tags, with string names and values
*
*
* The user-provided definition of an op template should capture a blueprint of an operation to be executed by
* a native driver. As such, you need either a statement or a set of params which can describe what
* specific type should be constructed. The rules on building an executable operation are not enforced
* by this API. Yet, responsible NB driver developers will clearly document what the rules
* are for specifying each specific type of operation supported by an NB driver with examples in YAML format.
*
* OpTemplate Construction Rules
*
* The available structural forms follow a basic set of rules for constructing the OpTemplate in a consistent way.
*
* - A collection of user-provided op templates is provided as a string, a list or a map.
* - All maps are order-preserving, like {@link java.util.LinkedHashMap}
* - For maps, the keys are taken as the names of the op template instances.
* - The content of each op template can be provided as a string or as a map.
*
* - If the op template entry is provided as a string, then the OpTemplate is constructed as having only a single
* statement property (in addition to defaults within scope).
* as provided by OpTemplate API.
* - If the op template entry is provided as a map, then the OpTemplate is constructed as having all of the
* named properties defined in the standard properties above.
* Any entry in the template which is not a reserved word is assigned to the params map as a parameter, in whatever structured
* type is appropriate (scalar, lists, maps).
*
*
*
*
* Example Forms
* The valid forms are shown below as examples.
*
* One String Statement
* {@code
* statement: statement
* }
*
* List of Templates
* {@code
* statements:
* - statement1
* - statement2
* }
*
* List of Maps
* {@code
* statements:
* - name: name1
* stmt: statement body
* params:
* p1: v1
* p2: v2
* }
*
* List Of Condensed Maps
* {@code
* statements:
* - name1: statement body
* p1: v1
* p2: v2
* }
*/
public abstract class OpTemplate implements Tagged {
private final static Gson gson = new GsonBuilder().setPrettyPrinting().create();
// TODO: coalesce Gson instances to a few statics on a central NB API class
public final static String FIELD_DESC = "description";
public final static String FIELD_NAME = "name";
public final static String FIELD_OP = "op";
public final static String FIELD_BINDINGS = "bindings";
public final static String FIELD_PARAMS = "params";
public final static String FIELD_TAGS = "tags";
/**
* @return a description for the op template, or an empty string
*/
public abstract String getDesc();
/**
* @return a name for the op template, user-specified or auto-generated
*/
public abstract String getName();
/**
* Return a map of tags for this statement. Implementations are required to
* add a tag for "name" automatically when this value is set during construction.
* @return A map of assigned tags for the op, with the name added as an auto-tag.
*/
public abstract Map getTags();
public abstract Map getBindings();
public abstract Map getParams();
public Map getParamsAsValueType(Class type) {
Map map = new LinkedHashMap<>();
for (String pname : getParams().keySet()) {
Object object = getParams().get(pname);
if (object != null) {
if (type.isAssignableFrom(object.getClass())) {
map.put(pname, type.cast(object));
} else {
throw new RuntimeException("With param named '" + pname + "" +
"' You can't assign an object of type '" + object.getClass().getSimpleName() + "" +
"' to '" + type.getSimpleName() + "'. Maybe the YAML format is suggesting the wrong type.");
}
}
}
return map;
}
public V removeParamOrDefault(String name, V defaultValue) {
Objects.requireNonNull(defaultValue);
if (!getParams().containsKey(name)) {
return defaultValue;
}
Object value = getParams().remove(name);
if (defaultValue.getClass().isAssignableFrom(value.getClass())) {
return (V) value;
} else {
return NBTypeConverter.convertOr(value,defaultValue);
}
}
@SuppressWarnings("unchecked")
public V getParamOrDefault(String name, V defaultValue) {
Objects.requireNonNull(defaultValue);
if (!getParams().containsKey(name)) {
return defaultValue;
}
Object value = getParams().get(name);
try {
return (V) defaultValue.getClass().cast(value);
} catch (Exception e) {
throw new RuntimeException("Unable to cast type " + value.getClass().getCanonicalName() + " to " + defaultValue.getClass().getCanonicalName(), e);
}
}
public V getParam(String name, Class type) {
Object object = getParams().get(name);
if (object == null) {
return null;
}
if (type.isAssignableFrom(object.getClass())) {
V value = type.cast(object);
return value;
}
throw new RuntimeException("Unable to cast type " + object.getClass().getSimpleName() + " to" +
" " + type.getSimpleName() + ". Perhaps the yaml format is suggesting the wrong type.");
}
@SuppressWarnings("unchecked")
public Optional getOptionalStringParam(String name, Class type) {
if (type.isPrimitive()) {
throw new RuntimeException("Do not use primitive types for the target class here. For example, Boolean.class is accepted, but boolean.class is not.");
}
if (getParams().containsKey(name)) {
Object object = getParams().get(name);
if (object == null) {
return Optional.empty();
}
try {
V reified = type.cast(object);
return Optional.of(reified);
} catch (Exception e) {
throw new RuntimeException("Unable to cast type " + object.getClass().getCanonicalName() + " to " + type.getCanonicalName());
}
}
return Optional.empty();
}
public Optional getOptionalStringParam(String name) {
return getOptionalStringParam(name, String.class);
}
/**
* Parse the statement for anchors and return a richer view of the StmtDef which
* is simpler to use for most statement configuration needs.
*
* @return an optional {@link ParsedStringTemplate}
*/
public Optional getParsed(Function... rewriters) {
Optional os = getStmt();
return os.map(s -> {
String result = s;
for (Function rewriter : rewriters) {
result = rewriter.apply(result);
}
return result;
}).map(s -> new ParsedStringTemplate(s,getBindings()));
}
public Optional getParsed() {
return getStmt().map(s -> new ParsedStringTemplate(s, getBindings()));
}
public abstract Optional> getOp();
public Map asData() {
LinkedHashMap fields = new LinkedHashMap<>();
if (this.getDesc() != null && !this.getDesc().isBlank()) {
fields.put(FIELD_DESC, this.getDesc());
}
if (this.getBindings().size() > 0) {
fields.put(FIELD_BINDINGS, this.getBindings());
}
if (this.getParams().size() > 0) {
fields.put(FIELD_PARAMS, this.getParams());
}
if (this.getTags().size() > 0) {
fields.put(FIELD_TAGS, this.getTags());
}
this.getOp().ifPresent(o -> fields.put(FIELD_OP,o));
fields.put(FIELD_NAME, this.getName());
return fields;
}
/**
* Legacy support for String form statements. This is left here as a convenience method,
* however it is changed to an Optional to force caller refactorings.
*
* @return An optional string version of the op, empty if there is no 'stmt' property in the op fields, or no op fields at all.
*/
public Optional getStmt() {
return getOp().map(m->m.get("stmt")).map(s->{
if (s instanceof CharSequence) {
return s.toString();
} else {
return gson.toJson(s);
}
});
}
public Element getParamReader() {
return NBParams.one(getName(),getParams());
}
/**
* @return the size of remaining fields from the op template and the params map.
*/
public int size() {
return getOp().map(Map::size).orElse(0) + getParams().size();
}
/**
* @return the map of all remaining fields from the op template and the params map.
*/
public Map remainingFields() {
Map remaining = new LinkedHashMap<>(getOp().orElse(Map.of()));
remaining.putAll(getParams());
return remaining;
}
public void assertConsumed() {
if (size()>0) {
throw new OpConfigError("The op template named '" + getName() + "' was not fully consumed. These fields are not being applied:" + remainingFields());
}
}
}
© 2015 - 2025 Weber Informatics LLC | Privacy Policy