LLM – Automatic creation of minutes from voice data (Whisper, ChatGPT)

LLM – Automatic creation of minutes from voice data (Whisper, ChatGPT) This time, I will talk about tools that use OpenAI’s Whisper and ChatGPT, which are very popular among LLMs (Large Language Models). This is a tool that automatically creates minutes from audio data such as MP3 and WAV, and I would like to briefly … Continue reading “LLM – Automatic creation of minutes from voice data (Whisper, ChatGPT)”

Python – How to transform columns in Pandas: how to use map and apply

Python – How to transform columns in Pandas: how to use map and apply This time, I will describe how to use map and apply, which are sometimes used in pandas. pandas is a library for data analysis in Python. pandas has various functions for preprocessing data, manipulating data, etc. This time, I will explain … Continue reading “Python – How to transform columns in Pandas: how to use map and apply”

Python – UNION like SQL in pandas

Python – UNION like SQL in pandas In the business, I often come across cases where data must be obtained directly from the data lake and preprocessed. Therefore, I think there are many opportunities to process large amounts of data with pandas. So, this time, I would like to record the case of UNION like … Continue reading “Python – UNION like SQL in pandas”

Python – INNER/LEFT/RIGHT JOIN like SQL in pandas

Python – INNER/LEFT/RIGHT JOIN like SQL in pandas When in the business, I often come across cases where data must be obtained directly from the data lake and preprocessed. Therefore, I think there are many opportunities to process large amounts of data with pandas. So, this time, I would like to record an INNER/LEFT/RIGHT JOIN … Continue reading “Python – INNER/LEFT/RIGHT JOIN like SQL in pandas”

Python – List string with pandas groupby

Python – List string with pandas groupby If you are doing business, you may have to acquire data directly from the data lake. Therefore, I think there are many opportunities to process large amounts of data with pandas for preprocessing. So, this time, although it is used less frequently, I would like to record a … Continue reading “Python – List string with pandas groupby”

Python – Scraping a Javascript page with Selenium

Python – Scraping a Javascript page with Selenium JavaScript is the most popular and well-supported client-side scripting language on the web today. It is used for user tracking information, form submissions without page reloads, multimedia embedding, and overall power-up of online games. Seemingly simple pages often contain multiple pieces of JavaScript. 1. Execute JavaScript in … Continue reading “Python – Scraping a Javascript page with Selenium”

Python – Scraping a website with BeautifulSoup

Python – Scraping a website with BeautifulSoup What is Web Scraping? In theory, web scraping refers to the act of collecting data by means other than direct programs using APIs (or humans using web browsers). This is usually accomplished by writing an automated program that queries a web server, requests data (in HTML or other … Continue reading “Python – Scraping a website with BeautifulSoup”

Python – read csv, tsv, excel files with pandas

Python – read csv, tsv, excel files with pandas I will focus on data input and output using pandas here, but there are many other libraries that can help you read and write data in various file formats. Inputs and outputs generally fall into several broad categories. Patterns for reading text files or more efficiently … Continue reading “Python – read csv, tsv, excel files with pandas”