Category: Coding

32 Posts

Easy Apache Airflow alerts

12 May 2023 Coding 1310

Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. It allows you to create complex data pipelines that can be executed on a schedule, triggered by an event, or manually. When self-hosting Airflow, it is crucial to keep track of what’s happening in your workflows to ensure everything is running smoothly. Without proper monitoring and alerting, it’s easy to miss critical issues that could cause your workflows to fail or produce incorrect results. These issues could be anything from a misconfigured task to a problem with your infrastructure or dependencies. By setting up alerts, you can...

Go to article

Parsing web sitemaps using JavaScript

15 Feb 2021 Coding 5725

Recently I came across an interesting project by Sean Thomas Burke called Sitemapper. This is a mini framework, which can be used to parse through sitemap XML files to get all included URLs. Such functionality is necessary when crawling through websites, as the sitemap (usually) holds an up-to-date list of all website URLs. In most cases this list should be enough when designing a crawler and you wouldn’t need to crawl manually the website and create a list of URLs. Sitemap parser: Sitemapper Sitemapper is a well-maintained and well-documented, open-source library offering the following features: Follows redirects Supports gzip sitemaps...

Go to article

HTTP status codes

27 Jul 2017 Coding Digital analytics Tag management 3769

If you are working with analytics you have to know what each HTTP status code means. Every piece of information travelling through the webs is also carrying a status code with it every time it travels. HTTP status codes are standard response codes given by web site servers on the Internet. The codes help identify the cause of the problem when a web page or other resource does not load properly. Every status code consists of 3 digits and belongs to one of 5 main groups. The first digit of the status code indicates the general type of response (the main group of the status...

Go to article

Excel function for MD5 hashing without VBA

04 May 2017 Coding Excel Web analytics 57619

When capturing PII data (Personally identifiable information) in GA or Adobe analytics, you need to make sure that the values captured are encrypted/hashed to respect the rules of these platforms. Otherwise you might have your account deactivated without any prior notice! A very common hashing algorithm is MD5. It produces a 128-bit hash value and it’s a one-way hashing algorithm, meaning that you cannot convert the hashed value back to the original one. (Keep in mind that MD5 hashes are only secure when using a unique input value, to prevent reverse lookup attacks e.g. using https://md5.gromweb.com/ ) To be able...

Go to article