upgraded openapi generator to v6.6.0#2532
Conversation
52d4f9f to
48f3727
Compare
5f9ba89 to
59f15fc
Compare
…as the fix already
|
/assign @roycaihw |
| """AdmissionregistrationV1ServiceReference - a model defined in OpenAPI""" # noqa: E501 | ||
| if local_vars_configuration is None: | ||
| local_vars_configuration = Configuration() | ||
| local_vars_configuration = Configuration.get_default_copy() |
There was a problem hiding this comment.
I recall we have something related to default config loading in the past. Does this impact the behavior?
There was a problem hiding this comment.
Let's announce this. The upstream change is: OpenAPITools/openapi-generator#8500
There was a problem hiding this comment.
added to Breaking change in CHANGELOG
| """Updates header and query params based on authentication setting. | ||
|
|
||
| :param headers: Header parameters dict to be updated. | ||
| :param querys: Query parameters tuple list to be updated. |
There was a problem hiding this comment.
Similar here. Is this a breaking change?
There was a problem hiding this comment.
added to Breaking change in CHANGELOG
| :return: The Auth Settings information dict. | ||
| """ | ||
| auth = {} | ||
| if 'authorization' in self.api_key: |
There was a problem hiding this comment.
added to Breaking change in CHANGELOG
|
Let's also add back the |
| # In the python 3, the response.data is bytes. | ||
| # we need to decode it to string. | ||
| if six.PY3: | ||
| r.data = r.data.decode('utf8') |
There was a problem hiding this comment.
@yliaog Do you know why this was dropped? Could it be related to the str response type cannot be decoded issue you saw?
During your test, could it be that
- The api_client.py decoded str once
- The rest.py decoded str for a second time, hitting the "cannot be decoded" error.
If that's the case, maybe the correct solution is:
- Remove the decode in rest.py (same as what you have in this PR)
- Let api_client.py to handle decode for all types.
There was a problem hiding this comment.
previously, as you can see in the deleted code here, response.data is bytes, bytes can be decoded to string
after the upgrade to v6.6.0, response.data is str, str cannot be decoded, as discussed here, https://share.google/aimode/t9bV2jOOKREzrlbRm
i think it is correct to drop the code here, as str cannot be decoded. and for the same reason, api_client.py should not try to decode str, in addition to file, and bytes there.
|
|
||
|
|
||
| WSResponse = collections.namedtuple('WSResponse', ['data']) | ||
| #WSResponse = collections.namedtuple('WSResponse', ['data', 'status', 'getheader']) |
expects it like RESTResponse.
… supports dict syntax
… generator to v6.6.0
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: roycaihw, yliaog The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
…-cluster When the scheduler runs in-cluster, kubernetes-python-client v36 changed model constructors to use Configuration.get_default_copy() instead of Configuration() (kubernetes-client/python#2532, OpenAPI Generator v6.6.0). So every V1Pod built after load_incluster_config() captures the global in-cluster Configuration, whose refresh_api_key_hook is a local closure (InClusterConfigLoader._set_config.<locals>._refresh_api_key). pickle cannot serialize a local closure, so putting a task's pod_override V1Pod on the executor's multiprocessing queue raises PicklingError and crashes the scheduler in a loop. This affects any in-cluster KubernetesExecutor deployment where a task sets a V1Pod pod_override, independent of the Airflow version; pinning the client below 36 is not viable because 35.x has a separate no_proxy regression. Reset local_vars_configuration to a fresh Configuration() on the pod_override (recursively) before queuing -- exactly what v35 model constructors produced. It carries no in-cluster auth hook so the pod is picklable, while keeping client_side_validation so the worker-side reconcile_pods setters still work. (Setting it to None instead breaks reconcile: model setters dereference self.local_vars_configuration.client_side_validation.) The pod keeps its V1Pod type through the queue, so run_next is unchanged.
…-cluster When the scheduler runs in-cluster, kubernetes-python-client v36 changed model constructors to use Configuration.get_default_copy() instead of Configuration() (kubernetes-client/python#2532, OpenAPI Generator v6.6.0). So every V1Pod built after load_incluster_config() captures the global in-cluster Configuration, whose refresh_api_key_hook is a local closure (InClusterConfigLoader._set_config.<locals>._refresh_api_key). pickle cannot serialize a local closure, so putting a task's pod_override V1Pod on the executor's multiprocessing queue raises PicklingError and crashes the scheduler in a loop. This affects any in-cluster KubernetesExecutor deployment where a task sets a V1Pod pod_override, independent of the Airflow version; pinning the client below 36 is not viable because 35.x has a separate no_proxy regression. Reset local_vars_configuration to a fresh Configuration() on the pod_override (recursively) before queuing -- exactly what v35 model constructors produced. It carries no in-cluster auth hook so the pod is picklable, while keeping client_side_validation so the worker-side reconcile_pods setters still work. (Setting it to None instead breaks reconcile: model setters dereference self.local_vars_configuration.client_side_validation.) The pod keeps its V1Pod type through the queue, so run_next is unchanged.
What type of PR is this?
/kind cleanup
What this PR does / why we need it:
upgraded openapi generator to v6.6.0, cleaned up unneeded patches
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: