Use in air-gap environments with network restriction and no internet [done]

[ycx91]

Now I understand that from tagui.py that it is downloading from

def setup():
    """function to setup TagUI to user home folder on Linux / macOS / Windows"""

    # get user home folder location to setup tagui
    if platform.system() == 'Windows':
        home_directory = os.environ['APPDATA']

Would it be good to change the code such that it does a check on whether the tagui folder exists and uses the existing folder rather than download a new one?

I ask this because I'm working in an environment where remote host downloading is not allowed, (yay web isolation, no curl either).
So by inspecting the code, I know that this folder is required and I've downloaded manually from https://github.com/tebelorg/Tump/releases/download/v1.0.0/TagUI_Windows.zip

[kensoh]

Thanks for sharing this use scenario!

Actually there is already checking if there is an existing installation in init(), and setup() would be once called to download TagUI only if an existing installation is not found.

# if tagui executable is not found, initiate setup() to install tagui
if not os.path.isfile(tagui_executable):

However, besides that, _tagui_delta() will be called to sync updated TagUI files through the internet from a GitHub https:// endpoint, for the particular TagUI for Python version. This will be called once whenever you install a new version of TagUI for Python.

For Windows, there is also automated downloading of vcredist_x86.exe, a supporting library from Microsoft, if it is not already installed on the computer. For macOS, there is automated installation of Python 3 SSL certs and OpenSSL.

I'm very interested to support usage of TagUI for Python on air-gapped environments or local installations without internet access. I'll think through over it what changes to the package can be made to reduce user friction in using the package in such environments.

In the meantime, do post here more of the restrictions, constraints and what is possible in your implementation environment. So that I can figure out which areas can be optimised, without increasing user friction for the primary use case of most users with internet access on their laptops.

[ycx91]

Hi Ken, I've dropped you a message on LinkedIn regarding further collaboration.
Ideally speaking, we should be able to package the dependency files in a dump place which also allows manual setup/downloads. So that when init and setup hits the error, we can prompt the user to perform the installation manually.
Would this be possible in the current setup or can there be a better approach?

[bfgoh]

Hi Ken and ycx91. I have actually message Ken on LinkedIn too, regarding installing TagUI that are completely sealed off from the internet. I was thinking if it is also possible to install TagUI at %temp% of Windows operating system.
As currently, I was thinking of implementing RPA on some of the repeated operations in my company. However, due to strict IT policy, I was unable to introduce TagUI to my company.

[kensoh]

The manual way will be for users to -

1. download and unzip TagUI zip file for their OS https://github.com/tebelorg/Tump/releases
2. download delta files from https://github.com/tebelorg/Tump/tree/master/TagUI-Python
   (delta files are those that have remarks 'stable delta files', including tagui.sikuli/tagui.py)
3. download Java JDK for their OS from Oracle website and Windows PHP dependency
4. put the files to folder %APPDATA%\tagui for Windows, ~/.tagui for macOS / Linux
5. for macOS / Linux, chmod -R 755 on the ./tagui folder for execute permissions
6. create a tagui_python_version file in above folder, eg tagui_python_1.9.1 for tracking delta
7. copy tagui.py and license file to the folder where TagUI for Python will be used
8. import tagui as t; t.init(); t.setup(); should work without problems

The question will be how much of above steps to automate or taken care of by the package. Beyond a certain point, the setup can't be automated. For eg downloading the appropriate Java JDK.

What will be good to know is, can you guys share how do you normally copy files to laptops on your restricted environment without internet? What OS do you use is such environments? This will be helpful to shape the approach to make TagUI for Python work offline, including installing without using pip.

[ycx91]

Thanks for the above steps. I'll try that out soon.
As you might have already guessed, the environment I'm working on is not fully air-gapped.
I am still able to download files but certain sites are restricted and scripts that directly try to do a connection without going through web isolation browsers are forcibly closed.
This means that I am unable to pip install directly but I can pip install from pypi.org by downloading the package and its dependencies first into a folder and then pip installing from that folder.
The typical OS used is Windows.
As for JDK, as it requires admin access to install, we have a list of pre-approved software that can be installed from a central repo.
So generally speaking, anything that requires admin access to do, its a no-go.

[kensoh]

I've tried doing the above steps on my macOS and it works without internet.

Refreshing feeling to switch off laptop wifi for the first time in many years.

For me, the high user friction part will be step 2 and 3. Delta files are not conveniently downloadable unless download whole tebelorg/Tump repo which is large. Otherwise is clicking to view raw to copy out. Step 3 going to Oracle website with so many options and may need to create ID to download is troublesome, but can't be overridden.

Will look out for @ycx91 and @bfgoh inputs on where is the user friction in setting up. So the package can be refined accordingly to minimise user friction in setting up for usage on air-gap envs. (without compromising or inconvenience to the primary users with internet of course).

[kensoh]

Adding on to above comment, @bfgoh setting up TagUI in %temp% folder for Windows is not reliable. Because Windows will periodically delete files from the temp folder and some files in tagui folders will be deleted randomly. That will cause TagUI operation to fail randomly. Not deterministic. Not good.

[alexdyysp]

I believe that it is an offline init problems for tagui.
I have tried a lot of way include to overwrite the download function to stop downloading and just unzip local file.But the work is too much, because running t.init() will download many files.So, I gave up, maybe it is impossible for me to use it in my work environment.
What's more, I think the problems is serious, A lot of computer environment in companies is not allowed to download in back-end, but I have to develop RPA in the same environment with companys so that I can easily share a ready-to-run rpa.exe with my colleagues.
At last, in my opinion, the offline init problems is necessary to be solved, if Tagui want to be more fashion then other rpa develop tools.

[kensoh]

How do you normally install programs and their files into laptops in your company network?

TagUI for Python tries to automate the process of downloading the required files instead of users manually downloading and doing the setup. Assuming there is a way to copy files (eg through thumbdrive or some medium), does the following work for your environments?

1. pip install tagui on normal computer with internet, run import tagui as t; t.init()
   (after this step there will be a ready-to-use folder for TagUI program files to be distributed)

2. copy folder to company computer - Windows %APPDATA%\tagui, macOS / Linux ~/.tagui

3. copy tagui.py & LICENSE.txt to where you want to use the tool (assuming pip can't use)

This will be the easiest way to use in air-gap environments, by using existing TagUI for Python t.init() (which calls t.setup()) to prepare the files on a computer with internet access. Before copying the folder over to the computer with restricted network access.

If above works for all the restricted network use cases for 3 of you, then I can create something like t.offline(). Which after running on a computer with internet access, will create a folder or zip folder that you can copy to your restricted computer to use.

[alexdyysp]

How do you normally install programs and their files into laptops in your company network?

Company has its own IT Center like a cloud center, only can install software it has stored.
It is a method to safe company data and local area network environment, especially in data sensitive company, just like my company.
And more, your main customers is in data sensitive industry, such as finance.

So I think it is necessary for you to add a .offline() function
Both my boss and me think your tagui is cool and full-functional

[kensoh]

Thanks for your feedback! Oh I don't have customers, TagUI and TagUI for Python are both personal projects that I started to make it easier to do digital process automation. So there is no cost to use or unlock full-features. TagUI is now maintained by AI Singapore, while I maintain TagUI for Python. But yes, a lot of RPA use cases will be in banking & finance.

Hi @ycx91 and @bfgoh , let me know if above method (#36 (comment)) works. Then I'll create a new function, maybe call it t.offline() or something. This function will generate a folder or zip file on your computer with internet access. This folder / zip file can then be copied to your company restricted network laptop to use TagUI for Python.