Problem with milo update

[johannwesely]

Hello,

i updated to the latest milo dev/1.0 commit and i am using digitalpetri uanodeset(latestVersion) as xml parser. Now i have two Problems first: my custom structure is shown as unknown extensionObject with some binaryData. Second: i have custom Object Types wich show an Error message when browsing with uaExpert. Can you give me a hint what possibly went wrong ?

[kevinherron]

I don't know from just the screenshot / errors.

"Unknown ExtensionObject" could mean that the datatypes aren't linking up correctly.

Try finding the DataType Node for the DataType indicated by that "ProcessingTimes" Node (navigate root -> types > base datatype > structure > etc...) and make sure it has the correct namespace index, name, and that it has a valid DataTypeDefinition attribute.

Also worth taking a look at the demo server, because it also loads a custom namespace (DataTypeTestNamespace) and uses types from it.

[kevinherron]

Also FWIW I have initiated the "release review" process with Eclipse Foundation and proposed May 1 as the release date for 1.0.0, with a 1.0.0-M1 release in the interim.

Whether I'm able to complete the process and actually release by then is anybody's guess, but the wheels are in motion for a stable 1.0.0 release :)

[johannwesely]

The DT_ProcessingTimesDataType seems to be OK i can browse it an definition looks good so far.

I wonder what i have to register in my DataTypeNamespace because in demoserver there is just

new DataTypeInitializer()
       .initialize(server.getNamespaceTable(), server.getStaticDataTypeManager()); 

i added some customTypeInitalizer which should register my custom Struct:

public class CustomTypeInitializer {
   public void initialize(NamespaceTable namespaceTable, DataTypeManager dataTypeManager) {
   	try {
   		registerStructCodecs(namespaceTable, dataTypeManager);
   	} catch (Exception e) {
   		throw new RuntimeException("DataType initialization failed", e);
   	}
   }

   private void registerStructCodecs(NamespaceTable namespaceTable, DataTypeManager dataTypeManager) throws Exception {
   	dataTypeManager.registerType(DT_ProcessingTimesDataType.TYPE_ID.toNodeIdOrThrow(namespaceTable), new DT_ProcessingTimesDataType.Codec(), DT_ProcessingTimesDataType.BINARY_ENCODING_ID.toNodeIdOrThrow(namespaceTable), DT_ProcessingTimesDataType.XML_ENCODING_ID.toNodeIdOrThrow(namespaceTable), DT_ProcessingTimesDataType.JSON_ENCODING_ID.toNodeIdOrThrow(namespaceTable));

   }
}

but it doesn't help either?

[kevinherron]

You would have always had to have been registering your types somewhere if this was working previously. Nothing would have changed there.

What does your code look like at the point where you encode an ExtensionObject?

What EncodingContext does your NodeSetNamespace / NodeSetAddressSpace return?

[johannwesely]

Hi im not sure if this is the encoding code you mean ?:

My DataTypeNamespace looks like this:

public class DataTypeNamespace extends NodeSetNamespace implements NamespaceBase {

	public static final String NAMESPACE_URI = ApplicationStrings.TOXCOMUA.getStr();

	Path path;

	public DataTypeNamespace(OpcUaServer server, Path path, OpcUaServerSetting configuration) {
		super(server, configuration.getApplicationUri());
		this.path = path;

	}

	@Override
	protected EncodingContext getEncodingContext() {
		return getServer().getStaticEncodingContext();
	}

	@Override
	protected List<InputStream> getNodeSetInputStreams() {
		URL xmlFilePath = ResourceUtils.getResource(DataTypeNamespace.class, "/ToxUaNodeset.xml", path);
		InputStream inputStream = null;
		try {
			inputStream = xmlFilePath.openStream();
			return List.of(inputStream);
		} catch (IOException e) {
			//TODO logger.error("Failed to load NodeSet", e);
		}

		return List.of();
	}

	public static DataTypeNamespace create(OpcUaServer server, Path path, OpcUaServerSetting configuration) {
		var namespace = new DataTypeNamespace(server, path, configuration);
		return namespace;
	}

	public static void namespaceInitalizer(OpcUaServer server) {
		ObjectTypeInitializer.initialize(server.getNamespaceTable(), server.getObjectTypeManager());
		VariableTypeInitializer.initialize(server.getNamespaceTable(), server.getVariableTypeManager());
		new DataTypeInitializer().initialize(server.getNamespaceTable(), server.getStaticDataTypeManager());
		new CustomTypeInitializer().initialize(server.getNamespaceTable(), server.getStaticDataTypeManager());

	}

	@Override
	public UaNodeManager getToxNamespaceNodeManager() {
		return getServer().getOpcUaNamespace().getNodeManager();
	}

	@Override
	protected boolean filterNamespace(String namespaceUri) {

		if (namespaceUri.equals("http://opcfoundation.org/UA/")) {
			return false;
		} else {
			return true;
		}


	}
}

[kevinherron]

That seems mostly okay, though that filterNamespace method is probably wrong... I would think it should be:

return NAMESPACE_URI.equals(namespaceUri);

but maybe it doesn't matter because there are no other namespaces in the NodeSet XML.

Probably want to make sure this isn't hiding a problem too:

//TODO logger.error("Failed to load NodeSet", e);

I was also asking about the places where you literally create/encode a value, e.g. a call to ExtensionObject.encode.

[johannwesely]

I don't have any places where i call ExtensionObject.encode. But I think i have multiple Namespaces in Nodeset Xml ?

[kevinherron]

How do your Nodes get their values? Are they static/hard-coded in the NodeSet XML?

[johannwesely]

I set the server values with :
Variant v = OpcUaDataVerification.getVariant(entry.getValue(), variableNode, server.getServer().getStaticEncodingContext()); variableNode.setValue(new DataValue(v));
in case of DT_ProcessingTimesDataType i do in OpcUaDataVerification
if (objectValue instanceof ToxUaStructure) { v = new Variant(objectValue);}

[kevinherron]

Ok I see.

Can you copy/pate the entire class file for one of your hand-coded custom structure types?

[johannwesely]

package com.toxware.opcua.custom;

import java.time.Duration;
import java.time.Instant;
import java.util.Objects;

import org.eclipse.milo.opcua.sdk.server.OpcUaServer;
import org.eclipse.milo.opcua.sdk.server.nodes.UaVariableNode;
import org.eclipse.milo.opcua.stack.core.UaSerializationException;
import org.eclipse.milo.opcua.stack.core.encoding.EncodingContext;
import org.eclipse.milo.opcua.stack.core.encoding.GenericDataTypeCodec;
import org.eclipse.milo.opcua.stack.core.encoding.UaDecoder;
import org.eclipse.milo.opcua.stack.core.encoding.UaEncoder;
import org.eclipse.milo.opcua.stack.core.types.builtin.DateTime;
import org.eclipse.milo.opcua.stack.core.types.builtin.ExpandedNodeId;
import org.eclipse.milo.opcua.stack.core.types.structured.StructureField;
import org.w3c.dom.Element;
import org.w3c.dom.Node;

import com.toxware.opcua.ToxUaStructure;
import com.toxware.opcua.common.AbsStructureSubClass;

public class DT_ProcessingTimesDataType extends AbsStructureSubClass implements ToxUaStructure {

	public static final ExpandedNodeId TYPE_ID = ExpandedNodeId.parse(String.format("nsu=%s;s=%s", "http://bmwgroup.com/UA/", "Types/DataTypes/BaseDataType/Structure/DT_ProcessingTimesDataType"));
	public static final ExpandedNodeId BINARY_ENCODING_ID = ExpandedNodeId.parse(String.format("nsu=%s;s=%s", "http://bmwgroup.com/UA/", "Types/DataTypes/BaseDataType/Structure/DT_ProcessingTimesDataType/BinaryEncoding"));
	public static final ExpandedNodeId JSON_ENCODING_ID = ExpandedNodeId.parse(String.format("nsu=%s;s=%s", "http://bmwgroup.com/UA/", "Types/DataTypes/BaseDataType/Structure/DT_ProcessingTimesDataType/JsonEncoding"));
	public static final ExpandedNodeId XML_ENCODING_ID = ExpandedNodeId.parse(String.format("nsu=%s;s=%s", "http://bmwgroup.com/UA/", "Types/DataTypes/BaseDataType/Structure/DT_ProcessingTimesDataType/XmlEncoding"));
	//
	public static final String F1 = "StartTime";
	public static final String F2 = "EndTime";
	public static final String F3 = "AcquisitionDuration";
	public static final String F4 = "ProcessingDuration";
	private Instant startTime = Instant.now();
	private Instant endTime = Instant.now();
	private Duration acquisitionDuration = Duration.ZERO;
	private Duration processingDuration = Duration.ZERO;

	public DT_ProcessingTimesDataType(Instant startTime, Instant endTime, Duration aquisitionDuration, Duration processingDuration) {
		this.startTime = startTime;
		this.endTime = endTime;
		this.acquisitionDuration = aquisitionDuration;
		this.processingDuration = processingDuration;
	}

	public DT_ProcessingTimesDataType() {
		this(Instant.now(), Instant.now(), Duration.ZERO, Duration.ZERO);
	}

	public static DT_ProcessingTimesDataType parseXml(Node node) {
		DT_ProcessingTimesDataType processingTimes = new DT_ProcessingTimesDataType();

		String startTime = node.getAttributes().getNamedItem(F1).getNodeValue();
		String endTime = node.getAttributes().getNamedItem(F2).getNodeValue();
		String acquisitionDuration = node.getAttributes().getNamedItem(F3).getNodeValue();
		String processingDuration = node.getAttributes().getNamedItem(F4).getNodeValue();

		processingTimes.setStartTime(Instant.parse(startTime));
		processingTimes.setEndTime(Instant.parse(endTime));
		processingTimes.setAcquisitionDuration(Duration.parse(acquisitionDuration));
		processingTimes.setProcessingDuration(Duration.parse(processingDuration));
		return processingTimes;
	}

	public static DT_ProcessingTimesDataType parseXmlElements(Node node, DT_ProcessingTimesDataType oldValue) {


		DT_ProcessingTimesDataType processingTimes = oldValue == null ? new DT_ProcessingTimesDataType() : oldValue;

		if (node instanceof Element nodeElement) {
			String startTime = getValueFromChild(nodeElement, F1);
			String endTime = getValueFromChild(nodeElement, F2);
			String acquisitionDuration = getValueFromChild(nodeElement, F3);

			if (!startTime.isBlank()) {
				processingTimes.setStartTime(Instant.parse(startTime));
			}
			if (!endTime.isBlank()) {
				processingTimes.setEndTime(Instant.parse(endTime));
			}
			processingTimes.setAcquisitionDuration(Duration.ofMillis(Long.parseLong(acquisitionDuration)));
			if (processingTimes.getStartTime() != null && processingTimes.getEndTime() != null) {
				processingTimes.setProcessingDuration(Duration.ofMillis(processingTimes.getEndTime().toEpochMilli() - processingTimes.getStartTime().toEpochMilli()));
			}
		}

		return processingTimes;
	}

	@Override
	public ExpandedNodeId getTypeId() {
		return TYPE_ID;
	}

	@Override
	public ExpandedNodeId getXmlEncodingId() {
		return XML_ENCODING_ID;
	}

	@Override
	public ExpandedNodeId getJsonEncodingId() {
		return JSON_ENCODING_ID;
	}

	public ExpandedNodeId getBinaryEncodingId() {
		return BINARY_ENCODING_ID;
	}

	@Override
	public StructureField[] getStructureField(OpcUaServer server) throws Exception {

		return new StructureField[] {getUtcTimeStructureField(F1), getUtcTimeStructureField(F2), getDurationStructureField(F3), getDurationStructureField(F4)};

	}


	public Instant getStartTime() {
		return startTime;
	}

	public void setStartTime(Instant startTime) {
		this.startTime = startTime;
	}

	public Instant getEndTime() {
		return endTime;
	}

	public void setEndTime(Instant endTime) {
		this.endTime = endTime;
	}

	public Duration getAcquisitionDuration() {
		return acquisitionDuration;
	}

	public void setAcquisitionDuration(Duration acquisitionDuration) {
		this.acquisitionDuration = acquisitionDuration;
	}

	public Duration getProcessingDuration() {
		return processingDuration;
	}

	public void setProcessingDuration(Duration processingDuration) {
		this.processingDuration = processingDuration;
	}


	public void actualizeValue(UaVariableNode node) {
		// This is empty because this method is only used in ZF Namespace
	}


	public static class Codec extends GenericDataTypeCodec<DT_ProcessingTimesDataType> {
		@Override
		public Class<DT_ProcessingTimesDataType> getType() {
			return DT_ProcessingTimesDataType.class;
		}

		@Override
		public DT_ProcessingTimesDataType decodeType(EncodingContext context, UaDecoder decoder) throws UaSerializationException {
			Instant startTime = decoder.decodeDateTime(F1).getJavaInstant();
			Instant endTime = decoder.decodeDateTime(F2).getJavaInstant();
			Duration acquisitionTime = Duration.ofMillis(decoder.decodeDouble(F3).longValue());
			Duration processingTime = Duration.ofMillis(decoder.decodeDouble(F4).longValue());
			return new DT_ProcessingTimesDataType(startTime, endTime, acquisitionTime, processingTime);
		}

		@Override
		public void encodeType(EncodingContext context, UaEncoder encoder, DT_ProcessingTimesDataType value) throws UaSerializationException {
			encoder.encodeDateTime(F1, new DateTime(value.startTime));
			encoder.encodeDateTime(F2, new DateTime(value.endTime));
			encoder.encodeDouble(F3, (double) value.acquisitionDuration.toMillis());
			encoder.encodeDouble(F4, (double) value.processingDuration.toMillis());
		}

	}

	@Override
	public boolean equals(Object o) {
		if (this == o) {
			return true;
		}
		if (o == null || getClass() != o.getClass()) {
			return false;
		}
		DT_ProcessingTimesDataType that = (DT_ProcessingTimesDataType) o;
		return Objects.equals(startTime, that.startTime) && Objects.equals(endTime, that.endTime) && Objects.equals(acquisitionDuration, that.acquisitionDuration) && Objects.equals(processingDuration, that.processingDuration);
	}

	@Override
	public int hashCode() {
		return Objects.hash(startTime, endTime, acquisitionDuration, processingDuration);
	}
}

[kevinherron]

Can you copy that ExtensionObject ByteString value from UaExpert and paste it here?

[johannwesely]

So is it somehow possible for you to have a remote debug session via teams or zoom?

[johannwesely]

the value is 0xf093e47c16aedb013083ec7c16aedb0100000000000000000000000000004a40

[kevinherron]

Ok that's different than the screenshot, but in either case they have seem to be indicating an encoding ID in the wrong namespace. This one is ns=0;i=147, with bogus encoding type to follow, so not really even a valid ExtensionObject... so something seems to be going wrong when encoding the ExtensionObjects on the server side.

[kevinherron]

I would set a breakpoint here:

milo/opc-ua-stack/stack-core/src/main/java/org/eclipse/milo/opcua/stack/core/encoding/binary/OpcUaBinaryEncoder.java

Line 677 in 9288b71

ExtensionObject extensionObject = ExtensionObject.encode(context, struct);

This is going to break any time a structured value is being encoded and sent to a Client.

Step through the encoding process and make sure nothing is throwing and all the type/encoding IDs are as expected for the UaStructuredType instance being encoded.

[johannwesely]

Hello we think we have found the Problem we used the wrong Encoding ID's as we use a Siome generated xml the ids are actually numbers and not strings. We changed that and it works.

But what do we do in the case there is no json Encoding ?
public static final ExpandedNodeId BINARY_ENCODING_ID = ExpandedNodeId.parse(String.format("nsu=%s;i=%s", "http://bmwgroup.com/UA/", "5038")); public static final ExpandedNodeId JSON_ENCODING_ID = ExpandedNodeId.parse(String.format("nsu=%s;s=%s", "http://bmwgroup.com/UA/", "??????????")); public static final ExpandedNodeId XML_ENCODING_ID = ExpandedNodeId.parse(String.format("nsu=%s;i=%s", "http://bmwgroup.com/UA/", "5039"));

and i wonder how this could work before ?

[kevinherron]

You can return ExpandedNodeId.NULL_VALUE when there is no encoding.

Previously UaStructuredType had default methods:

  default ExpandedNodeId getBinaryEncodingId() {
    return ExpandedNodeId.NULL_VALUE;
  }

  default ExpandedNodeId getXmlEncodingId() {
    return ExpandedNodeId.NULL_VALUE;
  }

  default ExpandedNodeId getJsonEncodingId() {
    return ExpandedNodeId.NULL_VALUE;
  }

But now it does not. I found that it was too easy to "forget" to implement one of those in some other places.

[johannwesely]

But we still have the TypeCache Problem you can see in the first picture marked as 2.

[kevinherron]

I don't know what that error means other than to take it at face value - try browsing those ObjectTypes and see if there is some kind of error during the browse or while reading the attributes.

[johannwesely]

Hello, when i Browse the ObjectType i have BadNodeIDUnknown for all the attributes of Objects and Variables underneath the browsed ObjectType? I just needed to browse to TopologyElementType to see this behaviour all children down the tree show the same behaviour. It is not only one ObjectType that is not working, the error is shown on all my custom objectTypes?

[kevinherron]

Do you have a NodeSet XML file(s) you can share? I think I'll have to try loading something and see if it's an issue with the uanodeset library, but the demo server only uses custom datatypes.

[johannwesely]

Hello should i send it to [email protected]?

[kevinherron]

Yes that will be fine.

[johannwesely]

Hello I have the same problem with the CTT Tool,
I get errors when trying to run a test against the Nodeset i sent you beforehand.

In Address Space Base i see similar NodeIds having problems like the ones showing error messages in UA Expert :

Also some custom Object types: (but i cant see for example OT_Carrier in CTT tool)

Any help is apprechiated
Greetings Johann

[kevinherron]

What CTT tests are causing this? I haven't been able to reproduce using the Address Space Base tests yet.

[kevinherron]

Okay I think I reproduced it with the namespace you sent me in email. It requires NodeIds that are using String identifiers instead of UInteger.

It does look like UaExpert and the CTT have the same bug in certain situations. I will open an issue against the CTT and bring it up with the group tomorrow.

I'll also look to see if there's something I can do as a workaround in Milo.

[kevinherron]

@johannwesely try #1451 when you can. It fixed the BadNodeIdUnknown errors for me. You may have other legitimate issues with your NodeSet to fix, though.