I used to work with SAP a lot and my employer decided to disable macros. I had some extremely tedious work (making and maintaining PN, woes of the MechE life) so I just wrote a script with SikuliX to automate the work by importing an excel sheet. It easily saved me hundreds of hours of work. And of course I followed the golden rule of automating things, never tell your manager.
My previous job as pharmacist in pharma company has a lot of grunt job that I tried to automate using only word and excel sheets. It possibly saves a lot of man hour.
Why only excel and word? Pretty much because I almost can't install anything in company laptop. excel Macros also prohibited. I was thinking to create a simple js automation app in browser if it gets attention.
I showed my little automation tools to my superior but it failed to get appreciation nor support.
In the end my colleagues asks for help to run the automation but they refuse to learn how to use it themselves. Bah.
> Why only excel and word? Pretty much because I almost can't install anything in company laptop. excel Macros also prohibited. I was thinking to create a simple js automation app in browser if it gets attention
FYI assuming you're on windows from the office suite, python msi installer allows you to install without admin escalation (install just for this user). Definitely a lot easier to automate jobs with python.
I used this years ago to setup a UI test automation framework for iOS apps in the very first days of iOS, before any commercial products were available. iOS was run in the emulator with Sikuli driving it.
It was very effective, Sikuli was a pleasure to work with and the ability to record the tests and then edit the scripts and make abstractions was invaluable. It was very effective.
Wow, I've been looking for something like this for a long time but all I could find was proprietary stuff or not-that-good free alternatives. I'm not being dramatic when I say I feel this is going to change my life, so thank you!
This is a pretty old project, started long before AI hype. It's still very cool, but I wonder why no one wrote something similar using all the advancements in CV that we have now.
Newer: UI.Vision is newer and I find it easier to use. A key difference is that UI.Vision supports text reading (via OCR) out of the box: https://ui.vision/rpa/x/desktop-automation
My key point was that there are newer alternatives (the OP asked for that). PyAutoGUI would be another newer alternative. So are Applitools and UIPath, but they are not open-source.
The challenge with Tesseract is that its OCR quality is (too) low for screenshots because screenshots are usually text on colored background. Tesseract is optimized for classical text document OCR. Of course, you can switch OCR engines with Sikuli (I did that before), but this requires touching the source code.
I've used this with Jython and I find it very cool and straightforward to use. I think it has potential for automated testing implemented by non-programmers. A potential downside is that I imagine running OpenCL takes a decent bit of GPU power.
It is also more flexible and quicker way to write automation scrips than most of the alternatives. It could be a good alternative to AHK, for instance.
Since my gaming days, I have always wondered whether there is software that could exactly track the bit stream of mouse moments at the frequency they are generated.
I was trying to automate complex jump movements in quake 2. Never did figure out how (it was not possible with Sikuli or other popular macro software at the time)
I worked on a government project in which we put a lot of work into making it difficult for automated attacks, after which our automated ui tests were not able to run, I left before a solution was found (if it was) but was arguing that SikuliX was the solution before I left.
I've used it to automate a desktop app in the past, a ruby script in the server side running in multiple windows instances doing GUI things… It was like having robots working for us…
Looks good. Plenty of RPA companies flogging this type of technology around at the moment under the guise of 'Intelligent Automation' or 'AI powered Automation'.
