A93: xDS ExtProc Support

[markdroth]

No description provided.

[eshitachandwani]

What does until its write to the ext_proc sidestream has cleared flow control. mean? Does it mean until we have written that message to side stream or does it mean until we have received window update for that message?

[markdroth]

We don't actually have to have it fully written to the sidestream, but we do have to have allocated enough flow control to send it at both the ext_proc layer and the HTTP/2 layer.

I've attempted to clarify the wording here.

[kannanjgithub]

Where are FlowControlInit initial window sizes to be set by the filter taken from? Should we add them to GrpcService config?

[markdroth]

I think each data plane implementation needs to figure this out on its own. A simple implementation would be to just use hard-coded values. But data planes may ultimately want some mechanism to dynamically tune these values when they are under memory pressure. In C-core, we plan to eventually use our ResourceQuota mechanism to do that.

I don't think it's appropriate to set these values from the control plane, because the control plane cannot know how much memory pressure a given data plane is under.

[eshitachandwani]

Is there a restriction here too similar to response received by the proc server that EOS without message should be set to true along with EOS and EOS should not be false if EOS without message is true ?

[eshitachandwani]

Hey @markdroth gentle ping on this !

[markdroth]

I'm not sure I fully understand your question. However, the intent here is that end_of_stream_without_message is relevant only if end_of_stream is true; if end_of_stream is false, then the ext_proc server shouldn't look at end_of_stream_without_message at all.

[ejona86]

We probably want to accumulate window updates and only send eagerly when at least half the initial window can be returned. That allows batching of the window update and greatly extends the amount of time available to piggy-back on an already-being-sent processing request.

[kannanjgithub]

We should also send client window updates eagerly if the sidesidestream's window just became 0 or negative, to avoid deadlocks.

[markdroth]

I've added a paragraph to the "Flow Control" section talking about options that implementations should consider, and I'm referring back to that here.

[markdroth]

As per offline discussion, I've removed the backend_service label from the metrics, because the ext_proc filter will run above load balancing, which means that we don't actually know the leaf cluster in a reliable way.

[ejona86]

This is not precise enough if a negative window update was sent. Do we want to just say ext_proc filter won't use negative window updates at this time?

[markdroth]

I've removed this paragraph. Given the fact that window updates can be negative, I don't think we can actually tell when the peer is behaving badly. This is unfortunate, but I think it's an inherent part of the trade-offs we've made in the flow control design.

[ejona86]

s/reads/consumes/ ? This "read" language shows up multiple places, but the important part is that it has been processed. Yes, I see the next paragraph says how we won't consider a "read complete" until it passes flow control, but that is nuanced as you aren't actually talking about "reading the message from the side stream" as that will complete and we're going to even read more messages from the side stream even without the "read completing". It's instead some higher-level conceptual model and it isn't clear whether each "read" is the actual read or the conceptual model. I had decided against making this comment, but then I read "One simple implementation would be that as the filter reads each message from the side-stream" and again I thought "people are going to get the wrong idea."

[markdroth]

I've tweaked the wording a bit to try to address this.

[ejona86]

Delaying returning half of the window does not cause delays, unless the window is already too small. We should not shy away from delaying half the window.

[kannanjgithub]

@markdroth A gentle reminder about this comment.

[markdroth]

@ejona86 I don't understand that comment. I think delaying a window update can cause delays. Consider the case where the sender needs to send three messages of 10 bytes each and the available flow control window is 15. The sender will be able to send the first two messages, after which the available window will be -5, so it will need to wait before sending the third message. If the receiver sends an immediate window update of 10 after it processes the first message, then the sender can immediately send the third message. But if the sender waits to send the update until after it processes the second message -- or even longer, if it wasn't going to send another message for some other reason anyway -- then I think that will cause a delay.

[ejona86]

You are assuming infinite throughput.

If the window is smaller than the BDP, then you can see a latency increase. But that is because the window is too small and throughput is artificially limited and not because of the delay. Basically, if the window is too small you can burst briefly, but that's it. Delaying the window update has no impact to the throughput. The problem is the window is too small, not the delay.

If the window of 15 was appropriate, then that means it takes at least ½ RTT to send 7.5 bytes on-the-wire, in which case the sender will get a window update before the sender consumes all 15 bytes of window, modulo aliasing because we process things message-by-message (e.g., four 8 byte messages would be fine.)

Note that sending updates too frequently can reduce throughput and increase CPU usage because of inefficient processing. Each window update is highly likely to be its own syscall and packet, because it is small.