• 11 Posts
  • 436 Comments
Joined 1 year ago
Cake day: September 1st, 2023
  • onlinepersona@programming.devtoPython@programming.dev•Multiple process or threads? And, why?English
    3·
    20 days ago

    IINM whether it’s “true” parallelism depends on the number of hardware cores (which shouldn’t be a problem nowadays). A single, physical core means concurrency (even with “hyper threading”) and multiple cores could mean parallelism. I can’t remember if threads are core bound or not. Processes can bound to cores on linux (on other OSes too most likely).

    So I suppose this is the preferred way to do concurrency, there is no async/await

    Python does have async which is syntax sugar for coroutines to be run in threads or processes using an executor (doc). The standard library has asyncio which describes valuable usecases for async/await in python.

    and you won’t use At “just” for a bit of concurrency. Right ?

    Is “At” a typo?

    We learn a little bit everyday. Thanks!

    You’re welcome :) I discovered the GIL the hard way unfortunately. Making another person aware of its existence to potentially save them some pain is worth it.

    Anti Commercial-AI license

  • onlinepersona@programming.devtoPython@programming.dev•Multiple process or threads? And, why?English
    181·
    21 days ago

    Python has a Global Interpreter Lock (GIL) which has been a bane and a boon. A boon because many basic types are thread-safe as actions happen in lock step. A bane because despite having multiple threads, there’s still a master coordinating them all, which means there is no parallelism but concurrency. Python 3.13 allows disabling the GIL, but I cannot say much to that since I haven’t tested it myself. Most likely it means nothing is really thread safe anymore and it’s up to the developer to handle that.

    So, in Python, using multiple threads is not a surefire way to have a performance boost. Small tasks that don’t require many operations are OK for threading, but many cycles may be lost to the GIL. Using it for I/O bound stuff is good though as the main python thread won’t be stuck waiting on those things to complete (reading or writing files, network access, screen access, …) . Larger tasks with more operations that are I/O bound or require parallelism (encoding a video file, processing multiple large files at once, reading large amounts of data from the network, …) are better as separate processes.

    As an example: if you have one large file to read then split out into multiple small files, threads are a good option. Splitting happens sequentially, but writing to disk is (comparatively) slow task that one shouldn’t wait on and can be dedicated to a thread. Doing these operations on multiple large files is worth doing in parallel using multiple processes. Each process will read a file, split it, and write in threads, while one master process orchestrates the slave processes.

    Of course, your mileage may vary. I’ve run into the issue of requiring parallelism on small tasks and the only thing that worked was moving out that logic to a cython and outside the GIL (terrible experience). For small, highly parallel operations, probably Python isn’t the right language and something like Rust should be explored.

    Anti Commercial-AI license

  • onlinepersona@programming.devtoPython@programming.dev•Core Python developer suspended for three monthsEnglish
    21·
    3 months ago

    So is that how courts work for you? You are guilty because you are accused?

    Lawyer: Your honor, the defendant is accused of having said despicable thing and showing repulsive behavior. I quote from a newspaper article that quotes from a group, that doesn’t provide a source: […]
    Judge: The accusations are proof of wrong-doing and thus are sufficient to convict this criminal of being a “libertarian bro who only thinks of themself and tries to hide behind ‘just focus on the code!’, thinking it’ll excuse right-wing behavior”.

    That’s not how it works. I doubt @savoy@lemmygrad.ml read the article. You just quote accusations and provide it “evidence”, “proof” or as “support of who he was”, and not a single link to any of such action displayed in the accusations.

    The article describes exactly this fear of being sanctioned in secrecy without no links to evidence and you’re doing exactly that, just like OP.

    Anti Commercial-AI license