Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upExecutor::this_worker_id() not working in larger programs #158
Comments
|
I am a bit confused. |
|
From a worker thread, as the logging I posted indicates. |
|
There is a detail here about shared objects that makes this less obvious. Apparently gcc aggregates thread_local setup to program / shared library boundaries. The two interacting translation units have to be across that boundary. Here's a test case that reproduces the problem: file1.cc:
file2.cc:
Build like this: Output is: g++ (Ubuntu 7.2.0-8ubuntu3.2) 7.2.0 |
|
@tomager Thanks for bringing this up! I wonder if there is any way to solve this without making taskflow non-header only. I recall in C++17 inline variable may be a good way to do. What is your take? |
|
So I can get file1/file2 to see the same static inline variable, but any use of thread_local doesn't want to play nice and uses the old non-inline static rules. Direct use of "static inline thread_local" doesn't compile (on gcc 7.2 anyway). The only workable solution I see is to use a static inline struct to initialize some thread local data with pthread/win32 apis rather than thread_local, like
That's a little bit of fuss but not too wacky, and keeps the library header-only. First step should be to try "static inline thread_local PerThread pt" on a more recent compiler and see if that works as is. |
|
More specifically, the patch below fixes the problem and keeps the library header only, using only the non-thread_local "static inline pt0". Handling win32 isn't much different I believe (ifdef on WIN32, use TlsAlloc, TlsGetValue, TlsSetValue). Thoughts?
|
|
Building with gcc head (g++ (GCC) 10.0.1 20200416 (experimental)) the following simpler patch works with naive use of "static inline thread_local PerThread pt". That doesn't have pthread/win32 dependency, but it will break the library for recent versions of gcc, and maybe all versions of clang/msvc. Those would need testing anyway. The manual tls method I posted earlier seems like the safer way to go, at least for the near future. Let me know your thoughts on how to proceed.
|
|
I definitely prefer the last solution of using inline variable, but as you said, it's supported only by C++17. The problem is NCVV is not yet there. I will keep this an open issue and see what I can do in the future. Perhaps the best solution for now is to resort the worker_id mapping to the client. |
|
If you're willing to have the client deal with worker_id, why not just make the -1 case an error (eg, throw I feel like that would be the most friendly for users today, and it's a trivial patch. In the future go with the c++17 method. |
Some logging at tf::Executor::_spawn:
spawn 0 _per_thread()=0x7f56323bf6f8 (thread 140008186705664)spawn 1 _per_thread()=0x7f5631b6e6f8 (thread 140008177985280)Now, within a task run from above Executor, here is logging from within this_worker_id:
this_worker_id _per_thread()=0x7f5631b6e658 (thread 140008177985280)Which results in:
exec.this_worker_id() = -1The thread number is the result of std::this_thread::get_id(). Note the varying &_per_thread() value above for the same thread 140008177985280.
What appears to be happening is that because _per_thread is inlined, it is duplicated multiple times in the program, and hence there are multiple instantiated PerThread structs per actual thread (one per instantiated _per_thread in the final link). Only the one touched by Executor::_spawn is initialized with the worker info.
Moving Executor::_per_thread() into a single translation unit fixes the problem. I would submit a patch for that, but it would render cpp-taskflow no longer a header-only library, so I am creating this issue instead. I am happy to generate a patch with guidance about preferred solution.