Robust GDScript Struct Implementation

[IphStich]

Structs Proposal

Central Philosphy

This proposal for Structs in Godot is taken with the following central philosophy in mind:

"Because we don't have Structs, we have been forced to make do"

What I mean by this is: Structs do not exist in Godot, and because they do not exist, much of Godot has been built around using Dictionaries and Resources instead. Everything from in-built engine functionality, through to the games themselves. However, Dictionaries lack a method for specifying a predetermined structure, which means losing out on safety and usability. As a result, many people opt to use Resources instead, however this runs into another problem that is: performance and memory. Which means gamedevs have been forced to choose between performance (by using a Dictionary) or ease-of-use and confidence of typing (by using a Resource) due to the lack of Structs in Godot.

It is my view that any implementation of Structs in Godot needs to respond to this and solve these problems. Which gives us some faily basic requirements to start from. Any implementation of Struct in Godot would need to do the following things:

• Be more or as performant as Dictionary.
• Be more or as useful as Resource.
• Be compatible with existing projects (or require only minimal updates to use).

Sounds pretty hard, right? How can one thing do all of that? Well, as you will see below, I have an idea I think will work...

What is required and expected of Structs in Godot?

As a starting place, I would like to, as much as possible, be able to replace the paramaters and returns for many in-built functions with Structs (with the added challenge of breaking as few projects as possible). This gives us a lot of specific requirements and guidance that can actually help us narrow down the possibilities and questions on how Structs should be implemented in Godot. In order to maintain compatability:

• Structs must be passed and assigned by reference.
• Structs must be implicitly castable both to and from Dictionary without issue.

This presents a bit of a problem. If part but not all of a project expects Structs and other parts Dictionaries, then wouldn't that mean that types are being changed, and wouldn't that break references or cause problems with un-typed variables?

My solution is that we need to therefore decide on the following set of requirements:

• Casting a Dictionary to a Struct, then back to a Dictionary, should always result in a value that is binary identical to the first Dictionary (that is: it is a reference to the same HashMap in memory).
• Regardless of how values are accessed, they should always be reading and writing to the same set of underlying values. (e.g. If I cast a Dictionary to a Struct with a known property of x:int, then I perform the operation struct.x = 4, then dict["x"] == 4 should be true.)
• The operator[] on a Struct should still be able to read-write the key-values of the HashMap in the original Dictionary (even if they aren't present as members of the Struct). Meanwhile, operations involving .subscript members on the Struct should be guaranteed to match the types specified by the Struct definition.
• Structs need to share some set of behaviours and methods with Dictionaries. As an example: many in-built functions return empty Dictionaries to communicate an un-successful operation (e.g. a raycast that collided with nothing), so we will need to ensure that there is parrity between the behaviour of an empty Dictionary and a state of Struct (e.g. .is_empty() == true and .size() == 0).

How do we do that? Well, we need to start by agreeing on a defition of what a Struct type really is. And in the implementation proposed here, that would be that a Struct type is a "signature" or minimum set of requirements. That is, anything that has properties with those names and matching types will satisfy the signature, and so for all intents and purposes is an instance of that StructType. Which means the following:

• The operation value is StructType returns true if (at the time of execution) every property defined by StructType can be accessed on value using .subscript syntax, and that they have the same types as those defined by StructType.
• If the operation var my_struct: StructType = some_value was performed, and some_value is StructType would have evaluated to false at time of execution, then the execution of any expression to get or set my_struct.property should result in an error.
• While var my_struct: StructType contains the value of some_value, the execution of any expression to get or set my_struct.property should result in an error if the value of my_struct.property does not match the type specified by the StructType signature, and otherwise should be no different than if some_value.property was executed.

So, basically, they enforce behaviour around get_named and set_named. Given GDScript uses ducktyping, this approach actually solves a lot of other potential problems we might have with different Struct solutions as well!

• If the target is an actual Dictionary, simply writing if target is StructType will cause the editor to give you code completion results for that Struct with .subscript expressions, without any changing of the actual data within that Dictionary! And because it knows the types of those members, the editor can even give you code completition results for them too! This gives us all of the advantages of something like a Resource, but remains relatively lightweight like a Dictionary!
• Any .subscript operations on the Struct will ensure that the signature of the Struct does not break. That is, the editor should be able to check the types of the expressions for assignments, flagging any instances where values cannot be implictly cast, and potentially making these lines safe operations now! Types can also be checked during execution as well!
• Even Objects can be cast to a StructType, so long as they satisfy the signature!

Sounds like you're describing Interfaces?

Well, I kind of am. But that's mainly to explain the reasoning behind the choices I've made. Because we need to maintain compatability.

At the end of the day, these struct definitions can actually be used to instantiate lightweight structs at run-time! Which means structs can be instanced in GDScript, and GDScript can make use of tightly-packed strongly-typed sets of data through Variant. These operate nearly identically to structs defined natively in c++, and can even make use of their constructors!

And I actually have a working prototype for this!

Basically, the definition of any struct defined in c++ can be easily defined and referenced using macros. Then these are wrapped in a container that allocates memory for that struct, and points to it along with the definition. The definition contains the names and types of each property, and specifies what functions to use when accessing them.

As a bonus, this is tightly integrated with Dictionary, allowing for operator[] executions on the Dictionary to prioritise and access the tightly-packed struct data. Meaning that structs can be effectively soft-cast to Dictionary, without the performance cost of actually applying the properties to the Dictionary as key-values! (And meanwhile, the Dictionary can also be used to house any additional key-values, which gamedevs might be in the habit of doing with the return of in-built funcitons that currently use Dictionary.)

So what's next?

Well, that's why I'm here.

I feel like this implementation is really promising, and I want to make sure that I get it to completion while working with the community and the Godot team.

And as a part of that, I really want to make sure the behaviour and syntax makes sense and works as expected.

What about X?

How can Structs be used and what's the syntax?

# Formal Struct type definition
struct StructType:
    var a: int = 42 
    var b: float
    var c: Variant

# Casting to struct
var dict := {a = 3, b = 23.2, c = false}
var ad: StructType = dict
var bd := StructType(dict)

# Constructing new struct instance
var my_struct_a := StructType{}
var my_struct_b: StructType = {}
# a and b are equivalent

# Construcing new struct instance (with values passed in)
var my_struct_c := StructType{b = 2.0}
var my_struct_d: StructType = {b = 2.0}
var my_struct_e: StructType = {"b": 2.0}
# c, d, and e are equivalent
# these use Dictionary syntax to assign values on execution
# the "Dictionary" specified is just syntactic sugar
# these will be compiled into individual assignment operations
# only works when assigning using this syntax (will not try to construct values when assigning with a Dictionary variable)

# Constructing new sruct instances (shorthand with [])
var my_struct_f := StructType[2] # will apply given values to properties matching the order as declared
var my_struct_g: StructType = struct[2] # where the type of struct is known, the struct keyword can also be used
# use of these is generally ill-advised, as unexpected behaviour may result if properties change order and they aren't updated in constructor uses
# however, use is appropriate for anonymous struct types


# Shothand Struct type definition
struct OtherStructType: x:String="ABC"; y:bool
struct OtherStructType {x:String="ABC", y:bool}

# Anonymous Struct type definition
var local_struct_a := struct{x:String="DEF", y:Resource}
var local_struct_b: {x:String, y:Resource} = struct["DEF"]


# Anonymous struct types are really useful to specify function returns
func my_function() -> struct{i:int, j:bool}:
    return {i = 3, j = true} # is implicitly cast to the return struct type and never stored as an actual Dictionary
    return struct[3, true] # this should also work

func my_other_function(my_param: Variant):
    var local := {k:int=23} # anonymous struct definitions are useful inside functions
    if my_param is {k:bool}: # anonymous struct definitions can even be used in `is` checks

        struct LocalStruct {j:float} # structs can even be defined and named locally to a function
        var str_array: Array[LocalStruct] # which can be useful if you need to process a lot of data of the same type

Notes:

• Default values are only used when constructing new instances.
• If a default value is not provided, normal GDScript rules apply to assign default values to these (e.g. int=0).
• All properties must be statically typed. If a gamedev wants a Variant type on their Struct, they need to explicitly specify it using : Variant.

[] vs {}

It has been suggested in other discussion to swap my use of [ ] above { }. I like that idea, and think it might be useful to better distinguish these from Dictionaries. I leave the decision ultimately up to the community.

When should two structs be considered equal?

Given that Godot still relies heavily on ducktyping, and that by this approach Structs are enforced signatures, it is my opinion that comparisons on structs should mirror the resulting operations on their underlying types. Except in cases of comparisons to any rvalue structs.

# -- For Dictionary

var a := {b = 23}
var b := {b = 23}
var c := {b = 23, c = 43}

struct ST {b:int}
var a_s: ST = a
var b_s: ST = b
var c_s: ST = c

print(a == b) # true
print(a == c) # false

print(a_s == b_s) # true
print(a_s == c_s) # false - because a == c is false

# comparisons to temporary struct instances only evaluate the struct properties
print(a == ST[23]) # true
print(c == ST[23]) # true
print(a == ST(c)) # true
print(c == ST(a)) # true


# -- For Object

var i := Node.new()
var j := Node.new()

i.name = j.name = "XYZ"

struct ZT {name:StringName}
var i_s: ZT = i
var j_s: ZT = j

print(i == j) # false
print(i_s == j_s) # false - because i == j is false
print(i == ZT(j)) # true

[KoBeWi]

Does that mean structs would be Dictionaries internally? The struct's fields are known, so storing the keys in each instance would be a waste.

[Ivorforce]

It would create a huge cost as opposed to compact storage, both in terms of storage space and data indirection.

[IphStich]

They aren't actually! :)

The actual struct data is allocated to a contiguous block of memory. Then I wrap that in a container that has a pointer to the data, and a pointer to the definition. There is also a pointer to Dictionary data, which will be filled only if a Dictionary was necessary by operator[] to "extend" the properties of the struct, or if the Struct was cast from a Dictionary.

Hopefully some code snippets will help to illustrate/explain.

class StructDefinition {
	struct PropertyInfo {
		StringName name;
		MemberAddress address;
		VariantReaderWriter::TypeInfo type;

		_ALWAYS_INLINE_ void * ptr(uintptr_t const& p_struct) const {
			return reinterpret_cast<void *>(p_struct + address);
		}
		_FORCE_INLINE_ Variant get(uintptr_t const& p_struct) const {
			return type->get(ptr(p_struct));
		}
		_FORCE_INLINE_ void set(uintptr_t const& p_struct, Variant const& p_value) const {
			type->set(ptr(p_struct),p_value);
		}

		PropertyInfo() : address(0), type{VariantReaderWriter::get_native_type_info<nullptr_t>()} {}
		PropertyInfo(StringName p_name, MemberAddress p_address, VariantReaderWriter::TypeInfo p_type) : name{p_name}, address{p_address}, type{p_type} {}
	};



// -- actual StructDefinition definitions
public:
	Vector<PropertyInfo> properties;
	StringName qualified_name;
	StructConstructor constructor;
	StructDestructor destructor;

	const size_t get_size() const;
	const bool test_signature(const Variant &p_value) const;
	const PropertyInfo *property_info(StringName const& p_property) const;
	const PropertyInfo *property_info(String const& p_property) const;
	const PropertyInfo *property_info(Variant const& p_property) const;

	StructDefinition(Vector<PropertyInfo> p_properties, StringName p_qualified_name, StructConstructor p_constructor, StructDestructor p_destructor)
	: properties{p_properties}, qualified_name{p_qualified_name}, constructor{p_constructor}, destructor{p_destructor}
	{}
};
class VariantStruct {

	friend class Dictionary;
private:
	void construct_new(StructDefinition *p_definition) const;
	void delete_struct() const;

	// (VariantStruct behaves similar to Dictionary)
	struct VariantStructPrivate {
		void *instance = nullptr;
		StructDefinition *definition = nullptr;
		DictionaryPrivate *extension = nullptr;
		SafeRefCount refcount;
	};
	mutable VariantStructPrivate *_p = nullptr;
	void _ref() const;
	void _unref() const;


public:
	_FORCE_INLINE_ bool is_valid() const {
		return _p != nullptr;
	}
	_ALWAYS_INLINE_ bool is_using_struct() const {
		return is_valid() && _p->instance != nullptr;
	}
	_ALWAYS_INLINE_ bool is_using_dictionary() const {
		return is_valid() && _p->extension != nullptr;
	}
};

void VariantStruct::construct_new(StructDefinition *p_definition) const {
	if (_p == nullptr) {
		_p = memnew(VariantStructPrivate);
	} else if (_p->instance) {
		delete_struct();
	}
	_p->definition = p_definition;
	_p->instance = memalloc(p_definition->get_size());
	p_definition->constructor(_p->instance, p_definition);
}

void VariantStruct::delete_struct() const {
	_p->definition->destructor(_p->instance, _p->definition);
	memfree(_p->instance);
	_p->definition = nullptr;
	_p->instance = nullptr;
}

[IphStich]

The Dictionary is only used to create properties that don't actually exist in the definition (so as to maintain compatability).

Even when the Dictionary is used, the actual struct and data persists, and is given preference for keys that match property names of the struct when accessing the dictionary (so the behaviour is identical to if the struct was converted to a Dictionary).

The only time the struct is deleted and all the properties are converted to key-values on the Dictionary is if an operation is performed on the Dictionary that is incompatible with this implementation (e.g. a call to getptr), or an operation is performed on the Dictionary thyat would break the struct signature (in which case the performance cost is virtually identical, as it doesn't really matter whether the struct was converted to a Dictionary early or later).

[vnen]

I personally see no point in keeping a Struct compatible to a Dictionary. They should be convertible between each other considering the existing systems, but not more than that.

• Casting a Dictionary to a Struct, then back to a Dictionary, should always result in a value that is binary identical to the first Dictionary (that is: it is a reference to the same HashMap in memory).

Why? There's really no use case for this and no one would expect it to work this way. I assume this is because of compatibility with existing API (so dictionaries can be replaced with structs) but this is not necessary. The current API constructs dictionaries on the fly anyway (for things like PropertyInfo), so nobody is keeping a reference to that, so they don't need reference equality.

• Regardless of how values are accessed, they should always be reading and writing to the same set of underlying values. (e.g. If I cast a Dictionary to a Struct with a known property of x:int, then I perform the operation struct.x = 4, then dict["x"] == 4 should be true.)
• The operator[] on a Struct should still be able to read-write the key-values of the HashMap in the original Dictionary (even if they aren't present as members of the Struct). Meanwhile, operations involving .subscript members on the Struct should be guaranteed to match the types specified by the Struct definition.

Same as above, why do we need something like this?

• Structs need to share some set of behaviours and methods with Dictionaries. As an example: many in-built functions return empty Dictionaries to communicate an un-successful operation (e.g. a raycast that collided with nothing), so we will need to ensure that there is parrity between the behaviour of an empty Dictionary and a state of Struct (e.g. .is_empty() == true and .size() == 0).

This is reasonable for compatibility. However, how can you have an "empty struct"? Even if you not set any field, they are still there. We'll probably need a different way of communicating a missed raycast, which will break compatibility anyway.

I see two problems here. Some Dictionary methods may not make sense (like size()). The other is more about the future. At some point we can consider structs and dictionaries completely different beings, at which they will have no reason to be compatible. So those methods should be immediately deprecated to convince people to move away from them. We have similar issues with enums being a dictionary and we are unable to optimize them further because people are using dictionary methods on them.

• If the target is an actual Dictionary, simply writing if target is StructType will cause the editor to give you code completion results for that Struct with .subscript expressions, without any changing of the actual data within that Dictionary! And because it knows the types of those members, the editor can even give you code completition results for them too! This gives us all of the advantages of something like a Resource, but remains relatively lightweight like a Dictionary!
• Any .subscript operations on the Struct will ensure that the signature of the Struct does not break. That is, the editor should be able to check the types of the expressions for assignments, flagging any instances where values cannot be implictly cast, and potentially making these lines safe operations now! Types can also be checked during execution as well!
• Even Objects can be cast to a StructType, so long as they satisfy the signature!

Seems like you're promising a runtime validation of the struct interface. I don't see how this is feasible (you would have to check each key/field to make sure they match in type and in name).

Also, casting objects to structs seems a bit too much. There's not really any use case for this, it just make the system more complex since you need to validate that they match in structure. And you cannot (or should not) treat them as the same reference, that would be even more chaos. Anyone who needs this type of conversion can do so manually.

Dictionaries and structs should be different types. So asking if dict is Struct should return false, meaning it wouldn't really work as described here. The completion would be happy to oblige, but at runtime the block wouldn't be executed.

---

Overall, I think this is trying too much in making things compatible to the point of almost making it a worse option. If we're going to have all that trouble it would be easier to create "dictionary interfaces" and call it that instead of "structs".

[IphStich]

I personally see no point in keeping a Struct compatible to a Dictionary. They should be convertible between each other considering the existing systems, but not more than that.

Why? There's really no use case for this and no one would expect it to work this way. I assume this is because of compatibility with existing API (so dictionaries can be replaced with structs) but this is not necessary. The current API constructs dictionaries on the fly anyway (for things like PropertyInfo), so nobody is keeping a reference to that, so they don't need reference equality.

This is not the reason behind having that as a goal in this implamentation. As you said, these are constructed as-needed.

There are countless functions in Godot that return struct results. These are usually converted into Dictionaries when sending the results to scripts. And it is a goal of this implementation to replace these.

Because Dictionaries are passed and assigned by reference, once somebody has queried the API and handled the result, what they do with that result is now up to them. It is possible that people are passing these Dictionaries around within their projects, using them as parameters for various functions, and typing them in doing so.

If we replace the results for all these functions, and don't maintain reference equality through casting, then this has the potential to cause major problems in countless Godot projects. Consider the following...

func funca():
    var a := <get raycast result>()
    if not a.is_empty():
        hit_success(a)

func hit_success(hit):
    preprocess_hit_data(hit)
    if hit["is_bad_guy"]:
        hurt_bad_guy(hit)

func preprocess_hit_data(result:Dictionary):
    result["is_bad_guy"] = result["collider"] is EnemyType

If we change the API to return a struct, then that means a and hit will be typed as structs. If we don't replicate some Dictionary behaviour and have a way do have an empty struct, then the first is_empty() will fail, and the code will need to be re-written.

Further, because hit will be typed as a Struct, that will mean that either the call to preprocess_hit_data will fail (if we disallow implicit casting) or that the reference will be broken and the behaviour will be unexpected.

Maintaining reference equality AND ensuring the underlying values are the same on both ends is key to allowing this behaviour to work without issue.

Seems like you're promising a runtime validation of the struct interface. I don't see how this is feasible (you would have to check each key/field to make sure they match in type and in name).

I am not.

Dictionaries will be validated and checked against a given StructType only during is and cast operations. Casts that fail should result in a falsey value, similar to when API functions return empty Dictionaries.

Aside from that, there are 2 different implementations I was considering for ongoing validation. Either, we can check that individual properties (not the entire Dictionary) conform to the signature type when get/set. Or we could add a flag to the Dictionary to tell it to notify the struct should the signature be broken. Or we could simply not check or enforce the signature on an ongoing basis.

I figured that because typed Dictionaries enforce the typing on (almost) every single read and write, that checking the properties match when you get or set them would be the ideal approach.

[KoBeWi]

We can deprecate Dictionary methods and add new ones that use structs. It will require changes if someone wants to move to structs, but the old code will not break and people can still use Dictionaries as before. The compatibility does not need to be baked into the struct system, it should simplify the implementation.

[vnen]

If we replace the results for all these functions, and don't maintain reference equality through casting, then this has the potential to cause major problems in countless Godot projects. Consider the following...

This is using types, so it will give a bunch of type errors. Sure, it's not compatible and people will have to update the code, but I don't think it's worth it maintaining a complex compatibility layer just for a few folks who might be doing odd stuff with it.

I believe we can come up with a compatibility layer that is independent on how structs work. Or we can do as KoBeWi said and just create new versions of the methods with structs.

Seems like you're promising a runtime validation of the struct interface. I don't see how this is feasible (you would have to check each key/field to make sure they match in type and in name).

I am not.

Dictionaries will be validated and checked against a given StructType only during is and cast operations. Casts that fail should result in a falsey value, similar to when API functions return empty Dictionaries.

That's exactly what I'm saying. Whenever someone do x is Struct in the code it will do a runtime validation of the interface.

It will have to do something for casting, but this can use whatever system is in place to just set the struct fields. Although IMO I would reconsider if casting Dictionary -> Struct should be allowed. If this is easy enough to do, then it should be a runtime error when failing, not a falsey value (I'm not even sure what that would look like). The other way is somewhat necessary but much easier since you can put anything into a dictionary (even if they are typed we can rely on the underlying type checking system).

[IphStich]

This is using types, so it will give a bunch of type errors. Sure, it's not compatible and people will have to update the code, but I don't think it's worth it maintaining a complex compatibility layer just for a few folks who might be doing odd stuff with it.

FWIW, I don't think the compatibility layer is that complex.

I feel inclined to presume that this perspective is a result of poor explaining on my part. I may be incorrect in that assumption. Please allow me to try and clarify some points, and feel free to tell me I'm wrong and that this was understood.

GDScript behaviour:

• Instances of Structs have reference type behaviour (similar to Objects, Arrays, Dictionaries)
• StructTypes can be used to guarantee type safety for .subscript operations
• StructTypes can be used to create Structs

Public c++ behaviour:

• Structs can be implicitly cast to VariantStruct (if they have a signature defined using the macro).

• VariantStruct is internally reference-counted (similar to Dictionary and Array).

• VariantStruct has methods for getting and setting properties, as well as querying the structure and such.

• VariantStruct can be implicitly cast to Dictionary.

• Where VariantStructs are reference equal, their casts to Dictionary will also be reference equal.

• Dictionary cannot be cast to VariantStruct.

• Dictionaries that were cast from VariantStruct can be queried for the VariantStruct details (including pulling a reference to the actual c++ native struct).

Private c++ behaviour

• DictionaryPrivate and VariantStructPrivate have pointers to each other.
• Dictionary and VariantStruct are friend classes of each other.
• Any read or write operation on Dictionary with a String or StringName key, will check if it has a pointer to VariantStructPrivate, and then if that key matches the name of a property, and if it does will perform the read or write on that property.
• Any other operation on a Dictionary that has a pointer to VariantStructPrivate and is incompatible with the signature, will cause a deconstruction of the struct instance.

[vnen]

Adding this in a separate comment to split the discussion.

Basically, the definition of any struct defined in c++ can be easily defined and referenced using macros. Then these are wrapped in a container that allocates memory for that struct, and points to it along with the definition. The definition contains the names and types of each property, and specifies what functions to use when accessing them.

This sounds very similar to current PR on structs godotengine/godot#82198. Is there anything different or are you implementing a similar thing?

As a bonus, this is tightly integrated with Dictionary, allowing for operator[] executions on the Dictionary to prioritise and access the tightly-packed struct data. Meaning that structs can be effectively soft-cast to Dictionary, without the performance cost of actually applying the properties to the Dictionary as key-values! (And meanwhile, the Dictionary can also be used to house any additional key-values, which gamedevs might be in the habit of doing with the return of in-built funcitons that currently use Dictionary.)

My response to this is similar to my other comment: why do we need to have this? Are people really getting a dictionary from the API and adding their own keys to pass it around? Even if they are, they should still be able to convert the struct into a dictionary and pass it around, no need to keep referential equality between different types.

Note that the current work on structs is stalled mostly because the system is too complex. It seems here you are proposing something even more complex, so it's quite difficult to accept as it is.

[IphStich]

This sounds very similar to current PR on structs godotengine/godot#82198. Is there anything different or are you implementing a similar thing?

AFAIK, this PR wraps struct members in an array of Variant? The implamentation I am working on stores them very close to how c++ does in memory. This makes it more lightweight, and in some cases potentially faster.

It also means very little processing has to be done to prepare this data to be read by scripts. The struct data is copied into a place in memory, and linked with a pointer to the actual definition. No messing around with arrays or Variant needed.

I want to be clear in saying that the solution I currently have meets all the requirements I mentioned in the initial post. All that I'm missing is to work on the GDScript syntactic behaviour.

[MarianoGnu]

Which means gamedevs have been forced to choose between performance (by using a Dictionary) or ease-of-use and confidence of typing (by using a Resource) due to the lack of Structs in Godot.

Small comment on this, i only use Resources if i intend saving and loading from disk, otherwise i use RefCounted, which are just a little smaller

[KeyboardDanni]

Which means gamedevs have been forced to choose between performance (by using a Dictionary) or ease-of-use and confidence of typing (by using a Resource) due to the lack of Structs in Godot.

A very strange situation we've found ourselves in if a Dictionary, of all things, is considered faster than using a custom Class.

• Be more or as useful as Resource.

This isn't why people typically want to use structs. To me, there's one very important use case: I want a function to return short-lived, structured information from a query, such as an OS or Physics query, without having to deal with all the usual costs associated with memory allocation, hashing, etc. The calling function gets the information it needs from the result, and then it gets cleaned up along with the stack. If the data needs to live longer, it can be "boxed" up inside a class object where it would live on the heap.

• Structs must be passed and assigned by reference.

That's not how structs work in C#. Structs are passed by value there, as if they live on the stack (though this is more of an implementation detail in the CLR, which uses a garbage-collected memory model). And some form of stack-based value is necessary if we want to fix intersect_ray() and friends and have it perform well without needing to reuse an existing result object by passing as an in/out parameter.

• Structs must be implicitly castable both to and from Dictionary without issue.

What's the use case here? If it's for compatibility, then we'd already break compatibility by changing the return type of the function. Better to make a new function called intersect_ray2 or similar and deprecate the old one.

It seems like most of this proposal is based around making said breaking changes to the existing API, and that's not how change control typically works with this stuff.

[vnen]

When doing structs as Array it was necessary to make them passed by reference, but since now the preferred approach seems to be creating a new Variant type, we can make it so it's always passed by value. I believe the main point in this proposal is making them compatible with Dictionary, which I do not think it's necessary.

[KeyboardDanni]

From what I can gather, the reason this proposal is trying to make structs compatible with Dictionary is because it wants to change PhysicsDirectSpaceState2D.intersect_ray() to return a struct instead of a Dictionary, which would cause a lot of breakage.

From that perspective, this proposal misses a few important points:

1. Classes are slow because they take a lot of time to construct, with each call to the returning function allocating memory for the object it returns.
2. To avoid heavy construction costs for a value that doesn't (usually) need to live long anyway, it can't be pass-by-reference because then something other than the stack has to be responsible for allocating (slow) and deallocating (or risking leaking) that memory.
3. Changing return types in an API is a bad idea and is why deprecation mechanisms exist.

[IphStich]

This proposal is primarily focused on discussing an approach to Structs which would allow for in-place replacement of the return types of various API functions that wouldn't result break any existing Godot project.

If the proposal is implemented, then the following behaviour will be exhibited in existing projects:

• If the gamedev statically types their variables: the return will be implicitly cast to Dictionary, and the behaviour will be identical.
• If the gamedev does not statically type their variables: the behaviour will be identical when using the operator[], and only uses of .subscript will cause an error.
• Meanwhile, performance will be boosted (because the struct is only ever deconstructed into a Dictionary if/when necessary).

(By "deconstructed into a Dictionary" I mean: the properties of the struct are copied as key-values to a Dictionary, and then the struct is deallocated from memory)

---

On the other hand, if the implementation of Structs from this proposal is integrated into the engine but the actual return types of these functions remains unchanged from Dictionary, we can still make use of the tight integration to silently improve performance (where users don't cause/force the struct deconstruction to Dictionary), while allowing some users to use type checking/casting to ensure type safety in their projects!

[MarianoGnu]

I believe the reason for easily casting structs from and to dictionary is for backward compatibility with all the methods we already have that takes or returns dictionary or arrays of dictionaries like get_property_list, wich is screaming to use a struct PropertyInfo instead

About passing structs as value instead of reference, i believe that is doable but the coast in memory allocation every time you pass the value around would be too high for no gain, since unlike other Variants, structs would not be 64bits (this is why arrays and dictionaries works as pointers to memory allocated in the heap, and accessed from the variant)

The way the feature is described, structs are just a wrapper to access the dictionaries we already have, but seems to be more than sugar sintax

I expected structs to be sugar sintax for dictionaries, just like enums (possibly forbidding to add functions to the struct, this is probably fine since they are meant to be just data containers)

[MarianoGnu]

From the point of view of upgrading a project from 4.5 to 4.x it is a potentially big change to enforce, this is why casting from and to dictionaries is proposed to be autimatic and out of the box, sure we can deprecate the old use, and force users to update their code, it is a commitmet that we have to decide to enforce or not, without this mechanism, users will have no choice.

Given the case, i am personaly strongly against of making an intersect_ray2() method only for backwards compatibility, i would preffer to cause a compile errors and make the users fix it. I would like to have the API as bloatless as possible, if anything, add an _intersect_ray_legacy() and make the upgrade tool change the old callbacks for the legacy ones, tagged as internal or something

[IphStich]

For those following this discussion, please know I have submitted a draft PR proof-of-concept.

godotengine/godot#108503

This implementation does not assume the Dictionary-compatability approach (which is the basis of this discussion), but it does make use of the underlying struct-definition approach I have been working on. If nothing else, I would like to know if such an approach is acceptable (even if Dictionary-compatability is decided against).

[eimfach]

Awesome work, also the syntax is clear and decent. I'd also stick with the shorthand [] and the {} for the literal syntax. Thank you !

[fteppe]

I have a question: would the proposal allow structs to be @export ? Make it possible to edit values of struct members in the editor (even nested struct ?) and to serialize them when part of a resource ?

[leandro-benedet-garcia]

I asked that in rocket chat and the answer was yes.