Back to skills
extension
Category: Data & AnalyticsNo API key required

使用qsv从终端配置文件并清理大型CSV数据集

在将大型CSV文件加载到下游分析或自动化工作流之前,对其进行检查、性能分析、标准化和差异比较。

personAuthor: user_3c6cb52ehubcommunity

使用qsv从终端配置文件和清理大型CSV数据集

Inspect, profile, normalize, and diff large CSV files before loading them into downstream analytics or automation workflows.

Prerequisites

qsv binary and CSV datasets

Installation

Use the upstream install or setup path that matches your environment:

  • cargo install zipsign
  • brew install qsv
  • conda install conda-forge::qsv
  • git clone https://github.com/1991513ccie-png/skills

Requirements and caveats from upstream:

  • | py<br>📇🔣 | Create a new computed column or filter rows by evaluating a Python expression on every row of a CSV file. Python's [f-strings](https://www.freecodecamp.org/news/python-f-strings-tutori...
  • 🔣: requires UTF-8 encoded input.

Basic usage or getting-started notes:

  • | describegpt<br>📇🗃️🤖🌐🪄📚⛩️ CKAN | <a name="describegpt_deeplink"></a>Infer a "neuro-symbolic" Data Dictionary, De...

  • | luau<br>📇🌐🔣📚 CKAN Luau | <a name="luau_deeplink"></a>Create multiple new computed columns, filter rows, compute aggregations and build complex...

  • | moarstats<br>📇🏎️ | Add up to an additional 55 statistical measures, including extended outlier, robust & bivariate statistics to an existing stats CSV file. ([example](docs/moarstats/NYC_31...

  • Source: https://github.com/1991513ccie-png/skills

  • Extracted from upstream docs: https://raw.githubusercontent.com/dathere/qsv/HEAD/README.md

Documentation

  • https://qsv.dathere.com