Make IndexTemplateRegistry project-aware

[nielsbauman]

Ensures the IndexTemplatesRegistry installs resources in every project in the cluster.

ES-10055

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[consulthys]

MonitoringTemplateRegistry changes LGT Stack Monitoring

[PeteGillinElastic]

A thought: We use this pattern a few times — I think I can see five instances in this class. I wonder whether an overload of ProjectResolver.executeOnProject which takes an additional Executor argument would be a good idea? Then executor.execute(() -> ...) could be replaced by projectResolver.executeOnProject(projectMetadata.id(), executor, () -> ...). It would remove one level of nested lambdas from this code, which might be a bit more readable.

I guess one thing that slightly muddies the water is that this overload would probably want to take a plain Runnable rather than a CheckedRunnable<E>.

I'm not sure about this, what do you think?

[nielsbauman]

Hmm yeah I also thought about this for some time. We also have ProjectResolver#projectClient:

elasticsearch/server/src/main/java/org/elasticsearch/cluster/project/ProjectResolver.java

Lines 94 to 113 in a6d9758

	/**
	* Returns a client that executes every request in the context of the given project.
	*/
	default Client projectClient(Client baseClient, ProjectId projectId) {
	// We only take the shortcut when the given project ID matches the "current" project ID. If it doesn't, we'll let #executeOnProject
	// take care of error handling.
	if (supportsMultipleProjects() == false && projectId.equals(getProjectId())) {
	return baseClient;
	}
	return new FilterClient(baseClient) {
	@Override
	protected <Request extends ActionRequest, Response extends ActionResponse> void doExecute(
	ActionType<Response> action,
	Request request,
	ActionListener<Response> listener
	) {
	executeOnProject(projectId, () -> super.doExecute(action, request, listener));
	}
	};
	}

which would be a good fit here, except that it creates a new client on every invocation, which feels very wasteful in this class, as there will be a lot of invocations - we created that method as a starting point with the plan to probably improve it sometime later.

I think I'll switch to use that method as it won't have any impact on stateful/single-project clusters. On MP clusters, it will make a difference, but it's more of an optimization - which is not the top priority right now.

Edit: done in 0889dc9

[PeteGillinElastic]

Nit: Can we make a stronger assertion, that calledTimesMap.keySet() should contain exactly the IDs from event.state().metadata().projects()?

As well as being a stronger assertion, I think this would make it easier to see what this method is doing.

[nielsbauman]

Good idea! Done in 4efb382