Pydantic Private Fields (or Attributes)

Let’s say we have a simple Pydantic model that looks like this: When you instantiate a Pydantic model – post = Post(**data) – all the required attributes defined must be passed, i.e., id, title, body. What if you had one or more of the following requirements: This would mostly require us to have an attribute […]

Virtualization, Virtual Machines and Hypervisors

Virtualization (in computing) is the process of creating one or more completely isolated “virtualized” environments inside a physical computer/machine. Each of these environments behaves like a completely isolated “real” computer by emulating softwares and hardwares like operating systems, storage (disk), networking, peripheral devices, etc. If you’ve SSH’ed into small remote servers hosted by cloud providers […]

Airflow Scheduler Parallelism (Concurrency)

Airflow allows configuring and tuning the parallelism/concurrency of tasks at various levels (like DAGs) and via different approaches (like Pools). But at the top level, the father of all knobs provided that controls the overall number of task instances that the entire cluster can run simultaneously (running state), is the scheduler’s (or more appropriately the […]