🚧Summaries For Lessons 24+ Are Still Work In Progress!

“Whoever is patient has great understanding…" Proverbs 14:29

Start Course

Video

Resources

Lesson Objectives

Lesson Summary

Homework

Questions

Lesson-27

/

python-course

/

27-organize-project-with-modules-and-packages

How To Organize Project with Modules and Packages

Description here...

Lesson-27

/

python-course

/

27-organize-project-with-modules-and-packages

How To Organize Project with Modules and Packages

Description here...

Lesson-27

/

python-course

/

27-organize-project-with-modules-and-packages

How To Organize Project with Modules and Packages

Description here...

Previous Lesson

Previous Lesson

Previous Lesson

Next Lesson

Next Lesson

Next Lesson

Lesson Objectives

Lesson Objectives

Lesson Objectives

- Learn how to organize Python Projects in multiple file
- Python Modules
- Python Packages
- Logic of Importing Custom Modules/Packages

Lesson Summary

Lesson Summary

Lesson Summary

📃Organize Python Projects Like a Pro

Once you start working on larger projects you'd want to separate your code into multiple modules and packages. This will help you to structure your code so it's easier to write and maintain it. And then you can bring it all together in a single file (e.g. main.py) to execute your final code.

And don't worry, it's not even complicated to do that.


Also, before we dive in, let's define a few things so it's easier to follow along.

📄Module - is a .py file that contains reusable functions, classes, variables that you can import into other scripts.

📁 Package - is a directory (folder) that contains other Modules or Nested-Packages to better organize your reusable code.

Now let's learn how to create and use them.

📂Start Simple - main.py

Let's start with a single main.py file and create a few pseudo-functions. They will just hold print statements so we can track when we execute them and then we'll start to move them around modules and packages to understand how to reuse them. 

print('📄Running main.py')

def generate_video():
    print('Generating Video🎦...')

def generate_images():
    print('Generating Images🖼️...')

def generate_audio():
    print('Generating Audio🔉...')

# Execute Functions
generate_video()
generate_images()
generate_audio()
print('📄Running main.py')

def generate_video():
    print('Generating Video🎦...')

def generate_images():
    print('Generating Images🖼️...')

def generate_audio():
    print('Generating Audio🔉...')

# Execute Functions
generate_video()
generate_images()
generate_audio()
print('📄Running main.py')

def generate_video():
    print('Generating Video🎦...')

def generate_images():
    print('Generating Images🖼️...')

def generate_audio():
    print('Generating Audio🔉...')

# Execute Functions
generate_video()
generate_images()
generate_audio()

At the moment it looks simple because we use abstract code. In reality, each function can take hundreds of lines and eventually your file can become a mess with all additional classes, functions and variables necessary for each one.

So it will be great to separate different functionality of your script into its own modules or packages.

So far our project is
📁 root
├─ 📄 gen_audio.py
├─ 📄 gen_images.py
├─ 📄 gen_video.py
└─ 📄 main.py

📄Split Into Modules

Now we're going to start separating functionality into separate modules. This will help us separate the necessary logic for creating audio, images and video.
Sounds logical so far?

And as a reminder:

📄Module - is a .py file th t contains reusable functions, classes, variables that you can import into other scripts.

So create multiple files like this:

📁 root
├─ 📄 gen_audio.py
├─ 📄 gen_images.py
├─ 📄 gen_video.py
└─ 📄 main.py

Now let's update each file as following:

print('📄Running gen_audio.py')

def generate_audio():
    print('Generating Audio🔉...')

generate_audio() # Execute Function
print('📄Running gen_audio.py')

def generate_audio():
    print('Generating Audio🔉...')

generate_audio() # Execute Function
print('📄Running gen_audio.py')

def generate_audio():
    print('Generating Audio🔉...')

generate_audio() # Execute Function
print('📄Running gen_images.py')

def generate_images():
    print('Generating Images🖼️...')

generate_images() # Execute Function
print('📄Running gen_images.py')

def generate_images():
    print('Generating Images🖼️...')

generate_images() # Execute Function
print('📄Running gen_images.py')

def generate_images():
    print('Generating Images🖼️...')

generate_images() # Execute Function
print('📄Running gen_video.py')

def generate_video():
    print('Generating Video🖼️...')

generate_video() # Execute Function
print('📄Running gen_video.py')

def generate_video():
    print('Generating Video🖼️...')

generate_video() # Execute Function
print('📄Running gen_video.py')

def generate_video():
    print('Generating Video🖼️...')

generate_video() # Execute Function

And finally, we can import all of these functions back so we can execute them directly from main.py file. To import anything from modules we need to reference the file name and then bring the functions, variables or classes that we need.

Here are some abstract examples:

from module import test   # Import Specific Thing
from module import *      # Import Everything
import module             # Import Module Itself 
from module import test   # Import Specific Thing
from module import *      # Import Everything
import module             # Import Module Itself 
from module import test   # Import Specific Thing
from module import *      # Import Everything
import module             # Import Module Itself

And here's how it's going to look inside our main.py file to bring functions from our new modules.

print('📄Running main.py')

# Import Functions from Modules
from gen_video import generate_video
from gen_audio import generate_audio
from gen_images import generate_images

# Execute imported Functions
generate_video()
generate_audio()
generate_images()
print('📄Running main.py')

# Import Functions from Modules
from gen_video import generate_video
from gen_audio import generate_audio
from gen_images import generate_images

# Execute imported Functions
generate_video()
generate_audio()
generate_images()
print('📄Running main.py')

# Import Functions from Modules
from gen_video import generate_video
from gen_audio import generate_audio
from gen_images import generate_images

# Execute imported Functions
generate_video()
generate_audio()
generate_images()

And this is what we get in the console - A Mess!

It looks like a mess but that's exactly what our code is supposed to do...

Let's break it down. We execute main.py file and then we start importing functions from our new modules. And when we import something from a module, we'll execute the code inside that module. therefore we can see multiple print statements from each file. And only then we execute these imported functions and get their print statements.

Now, we need to fix it ASAP and we can use __main__ statement for that.

Sign-Up For Future Updates✨

Be among the first people to hear about
New Python Courses or Useful Resources!

Once there's enough demand I might start a Python Newsletter
with even more Tips and Tricks to help you learn it better!


Want To Donate? Click here.

Sign-Up For Future Updates✨

Be among the first people to hear about
New Python Courses or Useful Resources!

Once there's enough demand I might start a Python Newsletter
with even more Tips and Tricks to help you learn it better!


Want To Donate? Click here.

Sign-Up For Future Updates✨

Be among the first people to hear about
New Python Courses or Useful Resources!

Once there's enough demand I might start a Python Newsletter
with even more Tips and Tricks to help you learn it better!


Want To Donate? Click here.

🎯 'Main' Statement

In Python we can execute a very special command to check if it's executed from inside itself, or it's just being imported. And it looks like this:

if __name__ == "__main__":
    print("We're inside main file."
if __name__ == "__main__":
    print("We're inside main file."
if __name__ == "__main__":
    print("We're inside main file."

💡It doesn't matter how your file is code. It's always the same line of code.

Also the main statement only runs code when file is run directly, not when it's imported. This is how you can control your code execution directly from a file.

So let's go back to our new modules (gen_images.py, gen_audio.py, gen_video.py) and add this special statement so we only print if we execute code from inside these files.

def generate_audio():
    print('Generating Audio🔉...')

if __name__ == '__main__':
    print('📄Running gen_audio.py')
    generate_audio() # Execute Function
def generate_audio():
    print('Generating Audio🔉...')

if __name__ == '__main__':
    print('📄Running gen_audio.py')
    generate_audio() # Execute Function
def generate_audio():
    print('Generating Audio🔉...')

if __name__ == '__main__':
    print('📄Running gen_audio.py')
    generate_audio() # Execute Function
def generate_images():
    print('Generating Images🖼️...')

if __name__ == '__main__':
    print('📄Running gen_images.py')
    generate_images() # Execute Function
def generate_images():
    print('Generating Images🖼️...')

if __name__ == '__main__':
    print('📄Running gen_images.py')
    generate_images() # Execute Function
def generate_images():
    print('Generating Images🖼️...')

if __name__ == '__main__':
    print('📄Running gen_images.py')
    generate_images() # Execute Function
def generate_video():
    print('Generating Video🖼️...')

if __name__ == '__main__':
    print('📄Running gen_video.py')
    generate_video() # Execute Function
def generate_video():
    print('Generating Video🖼️...')

if __name__ == '__main__':
    print('📄Running gen_video.py')
    generate_video() # Execute Function
def generate_video():
    print('Generating Video🖼️...')

if __name__ == '__main__':
    print('📄Running gen_video.py')
    generate_video() # Execute Function

Now the code that we put under if __name__ == '__main__': will only be execute when we run the file directly. This is important when you start working with modules/packages in Python.

It's also a great way to keep working inside each file individually with your local tests, and then when it's ready you can import it to the main file and local test won't be executed. Great for quick development.

And now here's how it looks:

⚠️ I highly recommend you to spend some time on this step and execute each file individually. This will help you better understand the logic by seeing what's being printed and what's not.

📂 Organize Code In Packages

Now you know how to organize code in modules but that won't be enough.

Imagine your program grows and you need multiple files for each part (Audio, Video, Image). On top of that you might be creating lots of local files and you want to keep everything nicely organized.

How? Well, Packages of course!

📁 Package - is a directory (folder) that contains other Modules or Nested-Packages to better organize your reusable code.

So for now create a simple Packages called generators and move your modules inside of it. Also create an empty __init__.py file and ignore it for now. We'll cover it in a moment.

You project structure should look like this:

📁 root
├─ 📄 main.py
└─📁 generators
├─ 📄 __init__.py
├─ 📄 gen_audio.py
├─ 📄 gen_images.py
└─ 📄 gen_video.py

Then if you try to execute your main.py file you'll get an error because it can't fine the modules in the previous place.

And that's correct. We need to modify out import statement so we bring modules from the right place. Change main.py to this:

print('📄Running main.py')

# Import Functions from Modules
from generators.gen_video import generate_video
from generators.gen_audio import generate_audio
from generators.gen_images import generate_images

# Execute imported Functions
generate_video()
generate_audio()
generate_images()
print('📄Running main.py')

# Import Functions from Modules
from generators.gen_video import generate_video
from generators.gen_audio import generate_audio
from generators.gen_images import generate_images

# Execute imported Functions
generate_video()
generate_audio()
generate_images()
print('📄Running main.py')

# Import Functions from Modules
from generators.gen_video import generate_video
from generators.gen_audio import generate_audio
from generators.gen_images import generate_images

# Execute imported Functions
generate_video()
generate_audio()
generate_images()

So the only difference is that we added the package name before the module name like generators.gen_video ...

Now it will work!

🤔What is __init__.py File?

Let's take a step back and look at __init__.py file we created. This is a special file you put inside folders to mark them as a Package. ON top of that it will get executed the first time your package is used.

So when you import 3 functions from your package, on the first import everything inside of __init__.py file will be executed. In general it's used for logging, printing or creating better import shortcuts.

For example:

print('Running from __init__.py file from generators Package.')

from .gen_audio import generate_audio
from .gen_video import generate_video
from .gen_images import generate_images
print('Running from __init__.py file from generators Package.')

from .gen_audio import generate_audio
from .gen_video import generate_video
from .gen_images import generate_images
print('Running from __init__.py file from generators Package.')

from .gen_audio import generate_audio
from .gen_video import generate_video
from .gen_images import generate_images

Notice the leading dots (from .audio...) that’s a relative import. It tells python "Look next to me inside this package" .

Now why do we import modules inside of __init__.py file? Well, that will allow us to import all these functions directly from the package without going into modules.

So now we can simplify imports in main.py like this:

# BEFORE:
from generators.gen_video import generate_video
from generators.gen_audio import generate_audio
from generators.gen_images import generate_images

# AFTER
from generators import generate_video, generate_audio, generate_images
# BEFORE:
from generators.gen_video import generate_video
from generators.gen_audio import generate_audio
from generators.gen_images import generate_images

# AFTER
from generators import generate_video, generate_audio, generate_images
# BEFORE:
from generators.gen_video import generate_video
from generators.gen_audio import generate_audio
from generators.gen_images import generate_images

# AFTER
from generators import generate_video, generate_audio, generate_images

How cool is that? Now it's so much easier to import you functions from a Package without trying to remember all module names.

🪺 Nested Pacakges

Now let's take it a step further.

Right now we have a single generator package. But what if our code for Audio, Video and Images will start to grow and we might need multiple files to better organize it?

We could just create lots of modules inside of generator and then import whatever is necessary in final modules, but that won't be clean. So, instead I'll create packages inside of a package. This will help me organize my code nicer.

Here's how it would look:

📁 root
├─ 📄 main.py
└─📁 generators
├─ 📄 __init__.py
├─ 📁 images
├ ├─ 📄 __init__.py
├ └─ 📄 gen_images.py
├─ 📁 video
├ ├─ 📄 __init__.py
├ └─ 📄 gen_video.py
└─ 📁 audio
├─ 📄 __init__.py
└─ 📄 gen_audio.py

💡Notice that we create __init__.py files for each nested-module too. They can be empty.

Now everything is very nicely organized and we can still import everything inside of main. We can either import directly from the module like:

from generators.video.gen_video import generate_video
from generators.video.gen_video import generate_video
from generators.video.gen_video import generate_video

Or we could go back to __init__.py file and modify local import there so we keep the same import in the main.py file.

from .images.gen_images import generate_images
...
from .images.gen_images import generate_images
...
from .images.gen_images import generate_images
...

⭐Pro-Tip: How To Find Source Code

Once your project starts to grow and you work with modules and packages it can take time to find the source code of various functions you re-use.

But there is a cool trick, you can select any function and then click on CTRL+B (in pyCharm) and it will open up the right file and take you to the right line where selected function was defined. This is a game changer when you need to find source code to make some changes.

Happy Coding!

🙋‍♂️ See you in the next lesson.
- EF

Happy Coding!

🙋‍♂️ See you in the next lesson.
- EF

Happy Coding!

🙋‍♂️ See you in the next lesson.
- EF

Homework

Homework

Homework

It's best for you to watch the full video lesson to better learn about Modules and Packages. And then use the code snippets to try it out yourself.

I tried my best to summarize everything in a written format but it was really hard to keep it easy to read because we work with multiple files. But I hope it was clear enough to follow along.

⌨️ Happy Coding!

⌨️ Happy Coding!

⌨️ Happy Coding!

Questions

Questions

Questions

If you have any questions, leave them in the YouTube Comments.

If you have any questions, leave them in the YouTube Comments.

If you have any questions, leave them in the YouTube Comments.

Have a Question?

Have a Question?

Have a Question?

Sign-Up For Future Updates✨

Be among the first people to hear about
New Python Courses or Useful Resources!

Once there's enough demand I might start a Python Newsletter
with even more Tips and Tricks to help you learn it better!


Want To Donate? Click here.

Sign-Up For Future Updates✨

Be among the first people to hear about
New Python Courses or Useful Resources!

Once there's enough demand I might start a Python Newsletter
with even more Tips and Tricks to help you learn it better!


Want To Donate? Click here.

Sign-Up For Future Updates✨

Be among the first people to hear about
New Python Courses or Useful Resources!

Once there's enough demand I might start a Python Newsletter
with even more Tips and Tricks to help you learn it better!


Want To Donate? Click here.

PS. Python can change your career and how you think about problems.
Be Careful 🙂

PS. Python can change your career and how you think about problems.
Be Careful 🙂

PS. Python can change your career and how you think about problems.
Be Careful 🙂

Sposored by LearnRevitAPI.com