feedback from release verification

[kevinjqliu]

Apache Iceberg version

None

Please describe the bug 🐞

Context: https://lists.apache.org/thread/s0m77324rn871vr082z0gtfshhkdtk8c

A few notes from my verification:

I ran into a number of issues with the integration tests including
Pandas wasn't installed with 'make install', but is required to run the tests
Running the tests doesn't work in a virtual environment because we use --user in the poetry install
Running the tests hit errors like 'OSError: [Errno 24] Too many open files'
Docker compose didn't shutdown after tests/failures

• Pandas wasn't installed with 'make install', but is required to run the tests
• Running the tests doesn't work in a virtual environment because we use --user in the poetry install (Remove user flag from poetry installation #2486)
• Running the tests hit errors like 'OSError: [Errno 24] Too many open files'
• Docker compose didn't shutdown after tests/failures (add docker cleanup to integration tests #2465)

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[geruh]

I can take a look at some of these!

[kevinjqliu]

Thanks @geruh !

[kevinjqliu]

Pandas wasn't installed with 'make install', but is required to run the tests

Pandas should be installed. Its declared as a poetry extra,

iceberg-python/pyproject.toml

Line 310 in a56795f

pandas = ["pandas", "pyarrow"]

and make install installs all the poetry extras,

iceberg-python/Makefile

Lines 72 to 75 in a56795f

	install-dependencies: ## Install all dependencies including extras
	$(POETRY) install --all-extras
	install: install-poetry install-dependencies ## Install Poetry and dependencies

I suspect this is a symptom of another issue.

[kevinjqliu]

Thanks @geruh i believe we address most of the issues above.