fix: fix DataFrame.from_records with data as an iterator

[terencehonles]

• Tests added: Please use assert_type() to assert the type of any return value

[cmp0xff]

Hi @terencehonles , thank you for the proposal. However, according to the Pandas documentation, the data parameter in DataFrame.from_records accepts Sequences, not Iterables. Could you please raise an Issue or Pull Request in pandas-dev/pandas regarding the documentation?

[terencehonles]

I'm not sure if I have time to follow up right now. I opened this PR because mypy is now complaining about some existing code I have (which I have silenced with an ignore statement). This is the pandas implementation and it does handle an iterator https://github.com/pandas-dev/pandas/blob/f6fa9b679006d08fa99d8c2b6dbd834ec60875d6/pandas/core/frame.py#L2240-L2263.

[Dr-Irv]

@terencehonles the docs are updated, so now @cmp0xff can review this PR and provide any feedback

[cmp0xff]

The changes are now in accordance with the Pandas documentation.

One minor comment: the idea in L4702-L4711 seems to be duplicated by the code snippet below. Could you remove the latter? Thanks.

pandas-stubs/tests/test_frame.py

Lines 4730 to 4739 in 2c603f6

	# Testing with list of tuples (instead of structured array for type compatibility)
	data_array_tuples = [(1, "a"), (2, "b")]
	check(
	assert_type(
	pd.DataFrame.from_records(data_array_tuples, columns=["id", "name"]),
	pd.DataFrame,
	),
	pd.DataFrame,
	)

[cmp0xff]

Hi @terencehonles , could you make the small change mentioned in #1376 (review)?

[cmp0xff]

Almost there!

[cmp0xff]

Sequence is a subset of Iterator. Also the type variable in SequenceNotStr does not have a default value, so pyright-strict would complain.

Suggested change

	| Sequence[SequenceNotStr]
	| Sequence[Mapping[str, Any]]
	| Iterator[SequenceNotStr]
	| Iterator[Mapping[str, Any]]
	| Iterator[SequenceNotStr[Any]]
	| Iterator[Mapping[str, Any]]

[loicdiridollou]

If I remove the sequence it causes many issues, it can’t assign list[tuple[int, str]] to the iterator. Let me know if you have an idea. I’ll fix the default type.

[cmp0xff]

Newer version has been released

Suggested change

	ty = "0.0.1a25"
	ty = ">=0.0.1a26"

[loicdiridollou]

I may add more ignore and report to ty project, it’s causing some issues with union type

[Dr-Irv]

I may add more ignore and report to ty project, it’s causing some issues with union type

My take on any type checker is that if they create a new version that causes our type checking to fail, and the others are still happy with the code, we pin the version of the "failing" type checker, and create a simple case and report it to the type checker's maintainers.

So in this case, you pin the version that works for ty, and hopefully create a simple case that illustrates the issue for them.

[loicdiridollou]

@cmp0xff What is happening is that Sequence[SequenceNotStr[Any]] and Iterator[SequenceNotStr[Any]] do not seem to be matching the same, [(1, "a"), (2, "b")] matches to Sequence but not to Iterator. If you have some insights