Make LwIP Core Locking available

[HamzaHajeir]

Make the implementation of LwIP core locking available:
https://github.com/espressif/esp-lwip/blob/a45be9e438f6cf9c54ec150581819c3b95d5af6b/src/include/lwip/tcpip.h#L52-L59

Important when dealing with Raw APIs

[me-no-dev]

please provide more info. Why, how, etc.

[HamzaHajeir]

For OS Mode, the threading of LwIP Raw APIs should be taken care of, APIs should never be called in parallel to lwip task operations (which happens by using an RTOS, in calling APIs by both main and tcpip tasks), can be solved by two possible methods:

1. Using of tcpip_callback().
2. Using LwIP Core Locking.

The nongnu website is down right now, so here's a reference to the raw file of the documentation regarding it:
https://github.com/espressif/esp-lwip/blob/5378fd84df498759d4ba231f6ca3a42d6aa0fb93/doc/doxygen/main_page.h#L162-L182

This method is used and tested in H4AsyncTCP the library I maintain after the passage of its author, where it shows high reliability under heavy testing in ESP32 environment (The 8266 too but no OS).

And here's the relevant Kconfig description in esp-idf component lwip:
https://github.com/espressif/esp-idf/blob/0479494e7abe5aef71393fba2e184b3a78ea488f/components/lwip/Kconfig#L37-L46

[Jason2866]

What's the need/benefit for the Arduino code? In other words real life use case?
This forked lib from me-no-dev does not need.

https://github.com/mathieucarbou/AsyncTCP
https://github.com/mathieucarbou/ESPAsyncWebServer

[HamzaHajeir]

Refer to my last comment: #186 (comment)

…

On Sat, Jul 13, 2024, 13:59 Jason2866 ***@***.***> wrote: What's the need/benefit for the Arduino code? In other words real life use case? — Reply to this email directly, view it on GitHub <#186 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AH3O7J5YJK34YCBNZ6L325LZMEB7ZAVCNFSM6AAAAABJ4CLF2OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRWHA2TKNRUHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Jason2866]

One! use case. Why should a setup be changed that may introduce side effects hitting all espressif32 Arduino users? My experience Lwip setup changes should be only done to fix a bug hitting most use cases. For special use cases there is the Arduino Lib Builder ;-)

[HamzaHajeir]

One! use case. Why should a setup be changed that may introduce side effects hitting all espressif32 Arduino users? My experience Lwip setup changes should be only done to fix a bug hitting most use cases. For special use cases there is the Arduino Lib Builder ;-)

I know that, and for that matter I've been using the builder for at least more than a year.

But per the lock feature, it has no side effects, as it's well ported to esp-idf:
https://github.com/espressif/esp-lwip/blob/a45be9e438f6cf9c54ec150581819c3b95d5af6b/src/include/lwip/tcpip.h#L52-L64
https://github.com/espressif/esp-idf/tree/master/components/lwip/port/freertos

BTW, the H4 Stack is completely Async, with complete reliable stack starting from TCP to the application (MQTT/HTTPS WebServer/WSS/HTTPS Client). I've a running MQTT that publishes 13kB payload every 3 seconds over TLS, alongwith running HTTPS WebServer/WSS/HTTP Client, with lots of other stress tests performed.

The Lock is needed to allow it for public use with the official Arduino Core for at least the plain TCP part now.

[Jason2866]

But per the lock feature, it has no side effects, as it's well ported to esp-idf

Famous last words :-)

So it has NO impact of Flash Usage, RAM usage. Every piece of code does run as before?

[HamzaHajeir]

But per the lock feature, it has no side effects, as it's well ported to esp-idf

Famous last words :-)

So it has NO impact of Flash Usage, RAM usage. Every piece of code does run as before?

Too little overhead due to the usage of the mutex, check sys_arch.c. Count the bytes :-)

[me-no-dev]

as an unforeseen consequence of adding matter to Arduino, core locking is now enabled (for the new 3.1.x versions). So I can close this now and you can give it a show on the 3.1.0-RC2 release

[HamzaHajeir]

as an unforeseen consequence of adding matter to Arduino, core locking is now enabled (for the new 3.1.x versions). So I can close this now and you can give it a show on the 3.1.0-RC2 release

Thanks for accepting the Core Locking config.

As it's not merged, I think there should be the esp_matter component (or CHIP) enables it?

Where can I see the built libs of 3.1.0-RC2? Note that the release does not provide the built sdk libs.

[me-no-dev]

https://github.com/espressif/esp32-arduino-lib-builder/releases/download/idf-release_v5.3/esp32-arduino-libs-idf-release_v5.3-a0f798cf.zip

[me-no-dev]

you could install from git and work that way. Just make sure you use the release/v3.1.x branch of Arduino

[HamzaHajeir]

Great, just found the config being checked.

[Jason2866]

Btw. enabling has side effects and broke code ;-) So your statement

But per the lock feature, it has no side effects, as it's well ported to esp-idf

is wrong

[HamzaHajeir]

Btw. enabling has side effects and broke code ;-) So your statement

But per the lock feature, it has no side effects, as it's well ported to esp-idf

is wrong

It has no side effects. The breaking code was due to core safety CONFIG_LWIP_CHECK_THREAD_SAFETY being checked, which BTW exposes a risk source of bugs.

Check:

• Crash with error message: "Required to lock TCPIP core functionality" arduino-esp32#10526
• fix(asyncudp): Fixes and implements tcpip thread locking arduino-esp32#10415