The Curious Path

prompt和context的区别

Fri, 14 Aug 2026 23:43:20 +0000

原文： https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models prompt只是context的一部分。

Finding your unknowns

Fri, 14 Aug 2026 23:04:30 +0000

原文：https://claude.com/blog/a-field-guide-to-claude-fable-finding-your-unknowns A field guide to Claude Fable 5: Finding your unknowns 总结：即便是最先进的大模型，也是需要有正确的使用方法，还是需要人来对整体流程进行把控，需要人理解项目相关的知识盲区（也就是风险点），并对这些知识做整理。

Quality of Earning Report笔记

Mon, 26 May 2025 18:03:23 +0000

# Abbr M&A - Mergers and Acquisitions GAAP - Generally Accepted Accounting Principles, 是指一套在美国使用的会计原则、标准和惯例。

AI-Powered Due Diligence Platform

Mon, 26 May 2025 03:45:17 +0000

# 🛠️ 1. Core Product Modules ## 📥 A. Data Ingestion Layer This layer collects, cleans, and unifies all deal data. Inputs Accepted: - Accounting software (QuickBooks, Xero, NetSuite) - Bank & credit card statements (via Plaid or file upload) - POS exports (e.g., Square, Toast, Clover) - Payroll providers (Gusto, ADP, Paychex) - Tax returns / W-2s / 1099s (PDF extraction + OCR) - Broker CIMs (to extract seller-provided claims) Features: - Smart mapping of chart of accounts to standard LMM schema - OCR + LLM-based auto-classification of scanned PDFs - Historical trend normalization (standardizes fiscal year vs calendar year) - Reconciliation engine: Ensures consistency between P&L, bank deposits, and cash on hand ## 🧠 B. LLM-Powered Analysis Engine The heart of the platform—your AI due diligence analyst. Capabilities: 1. Financial Normalization & Adjustments - Owner compensation benchmarking (flag excessive salaries) - Detection of owner perks/addbacks (cars, travel, family on payroll) - One-time / non-recurring event detection (lawsuits, one-off projects) - Revenue recognition alignment (cash vs accrual) 2. Variance & Trend Analysis - Monthly / YoY revenue and margin swings with LLM-generated commentary - Seasonality detection - Expense trend outliers (e.g., sudden jump in subcontractor costs) 3. Cash Flow & Working Capital - 13-week cash flow snapshot - NWC normalization for deal structuring - Inventory/purchasing cycle risk flags 4. Customer & Vendor Concentration - Top customer/vendor reliance by % - LLM summarizes risk from churned customers - Lifetime value and cohort revenue retention if CRM data is provided 5. Labor & Operational Analysis - Payroll breakdown by department - Staff-to-revenue ratio benchmarking (e.g., techs per $1M HVAC revenue) - Compensation benchmarking vs BLS or industry averages ## 📊 C. Output: AI-Generated Due Diligence Report Deliverables: - PDF + Interactive Web Report - Standard sections: - Executive Summary - Financial Overview (Revenue, COGS, SG&A, EBITDA) - Key Adjustments & Addbacks - Working Capital & Cash Flow Snapshot - Customer/Vendor Analysis - Risk Flags Summary - Red Flag Watchlist - Dynamic footnotes: Every metric includes LLM-generated rationale - Toggle view: “Broker View” vs “Buyer View” vs “Bank View” to tailor emphasis ## 📈 D. Benchmarks & Valuation Module (Optional Add-On) - Compares business KPIs to industry averages (NAICS + zip code) - Suggests valuation range (3x–5x EBITDA, etc.) based on risk-adjusted metrics - Creates a pre-filled investment memo with comps + deal notes - Flagging system: - 🔴 Red: Customer concentration, cash-based revenue, no controls - 🟠 Yellow: Addbacks >15% of EBITDA, weak gross margins - 🟢 Green: Clean books, minimal owner involvement, recurring revenue # 🔐 2. Security, Accuracy & Explainability ## A. Explainability Layer - All LLM-generated insights are linked to source docs - Allows users to click on a number or sentence and see: - What doc it came from - The prompt that generated it - LLM confidence score (0–100%) ## B. Reviewer Console - Human analysts or advisors can review and override any section - Edits logged for audit trail (critical for trust with banks or LPs) # 🤖 3. Integrations - Accounting: QuickBooks, Xero, NetSuite - Banking: Plaid, MX - POS: Square, Clover, Shopify - Payroll: Gusto, ADP, Paychex - CRM (optional): HubSpot, Salesforce - Brokerage platforms: Axial, Acquire.com (for future deal flow ingestion) # 🎯 4. Target Use Cases Use Case | Persona | Outcome -- | -- | -- Pre-LOI red flag review | Searcher | Spend <$1K to screen out bad deals Post-LOI due diligence | PE Associate | Replace $30K QoE with faster insights Broker-side pre-listing | M&A Advisor | Improve CIM quality & credibility Financing partner prep | SBA lender or equity co-investor | Faster underwriting with standardized KPIs # 📌 5. Example Scenarios HVAC Company ($3M EBITDA, cash-based) → System detects 30% cash revenue unreported on books, large owner addbacks, top 2 clients = 50% revenue Multi-location Med Spa ($1.2M EBITDA) → Flag: Owner has 2 Porsches on P&L, rent above market rate on affiliate-owned RE Franchise Restaurant ($4.5M Rev) → System extracts POS data, flags declining AUV and labor cost creep vs category median # Reasons for QofE The primary reason for conducting a QofE is its direct impact on the purchase price. It evaluates 3 critical factors, helping buyers assess if the business price is reasonable. - Enterprise Value Determined by taking a multiple of EBITDA or similar financial measures, and analyzing the financials to determine if that financial measures is accurately reported as well as any key factors that impact the multiple - Net Debt Assessment of debt like items is done to highlight any potential reductions to Enterprise Value and future cash flows - Net Working Capital If less than a normal level of working capital is delivered at the closing of a transaction the buyer may be required to contribute additional capital in order to fund operations. A comprehensive analysis of the business is needed to ensure adequate working capital is delivered at closing. Advises self funded searchers, traditional searchers, private equity groups, family offices, and strategic acquirers in evaluating acquisitions with transaction values up to $50 million. ## Industries HVAC Government Contracting Professional Services Retail Landscaping E-Commerce Healthcare Manufacturing Plumbing Trucking Restaurants Real Estate/Construction Marketing Agencies Electrical Contracting ## Is QofE different than an audit? An audit is different from a QoE in a variety of ways. For one an audit is a regulated procedure that has specific guidelines for how it needs to be completed. A QoE is a consulting engagement that can technically be done by anyone although they are typically done by CPAs. An audit is much more balance sheet focused while a QoE is more focused on the income statement. The audit is looking to confirm that the financials comply with GAAP while a QoE helps a buyer understand the financial performance of the company that can be projected into the future as well as the key operating metrics. On deals where audited financials are available the audit is complementary to the QoE. [Understanding QofE](https://corporatefinanceinstitute.com/resources/valuation/quality-of-earnings-report/) 。

Facebook Marketing API resources

Sun, 30 Mar 2025 04:15:54 +0000

If you want to manage the Facebook Ads with API, you should check the marketing API. I'm thinking about build MCP server for facebook ads management. The [document](https://developers.facebook.com/docs/marketing-api/) is not friendly for the new comers, especially the auth token setup, there is no screenshot, and outdated, and both the doc and the pages are messy with lots of stuffs there. Luckily I found this YouTube video '[Facebook Graph API: A Full Guide for Marketers and Non-Techies](https://www.youtube.com/watch?v=IFM3Otvb7So)' very helpful. And checkout [Facebook API Tutorial: Graph API, Access Token and Developer Documentation Explained](https://www.youtube.com/watch?v=to4uTxSNo6Q) Once you watched the videos, you can check https://www.postman.com/meta/facebook-marketing-api/overview and play with it. And auotmation with make, check video https://www.youtube.com/watch?v=MjzUlHcuUtA, you can download the template at https://neilstephenson.beehiiv.com/p/video-tutorial-automated-facebook-ads-creative-testing. Another related is [ads library API](https://www.facebook.com/ads/library/api/), but it's not used to manage your Ads, it's providing your Ads stats, as you can see https://www.facebook.com/ads/library/report, and do some search at https://www.facebook.com/ads/library/.。

MTLS Setup with Hitch and Squid

Tue, 11 Mar 2025 22:48:21 +0000

# Install Squid Use Linux machine to install squid ``` sudo apt update sudo apt install squid ``` Config squid, change '/etc/squid/squid.conf' and allow all traffic ``` # And finally deny all other access to this proxy #http_access deny all http_access allow all ``` Restart squid: ``` sudo systemctl restart squid ``` Verify squid works (10.231.249.222 is the ip address of the machine): ``` curl -x http://10.231.249.222:3128 https://myip.ipip.net ``` # Install Hitch Then install Hitch, old version may not support MTLS, so just grab the latest one: ``` wget https://hitch-tls.org/source/hitch-1.8.0.tar.gz tar zxvf hitch-1.8.0.tar.gz sudo apt-get install libev-dev libssl-dev automake python3-docutils flex bison pkg-config make -y cd hitch-1.8.0 ./configure;make;sudo make install ``` Create Hitch Config '/etc/hitch/hitch.conf' as below: ``` frontend = { host = '*' port = '8443' } backend = '[127.0.0.1]:3128' # 6086 is the default Varnish PROXY port. workers = 4 # number of CPU cores daemon = off # We strongly recommend you create a separate non-privileged hitch # user and group user = 'hitch' group = 'hitch' # Enable to let clients negotiate HTTP/2 with ALPN. (default off) # alpn-protos = 'h2, http/1.1' # run Varnish as backend over PROXY; varnishd -a :80 -a localhost:6086,PROXY .. # write-proxy-v2 = on # Write PROXY header # Only support strong ciphers and TLSv1.2, TLSv1.3 tls-protos = TLSv1.2 TLSv1.3 ciphers = 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:!aNULL:!eNULL:!EXPORT:!LOW:!MD5:!DES:!3DES:!RC2:!RC4:!PSK:!kDH' syslog = on log-level = 2 # The path to the combined server certificate and its private key. pem-file = '/etc/hitch/cert/server.pem' ``` Create user 'hitch': sudo useradd -r -U -s /bin/false hitch # Test TLS Create server certificate and private key: ``` mkdir cert cd cert openssl req -newkey rsa:2048 -nodes -keyout server.key -x509 -days 365 -out server.crt -subj '/CN=yourdomain.com' -extensions SAN -config <(cat /etc/ssl/openssl.cnf <(printf '\n[SAN]\nsubjectAltName=DNS:yourdomain.com,DNS:www.yourdomain.com')) cat server.key server.crt > server.pem sudo mkdir -p /etc/hitch/cert sudo cp server.pem /etc/hitch/cert/ ``` Add domain name in '/etc/hosts': ``` 10.231.249.222 www.yourdomain.com ``` Add the Self-Signed Certificate to the Trust Store: ``` sudo cp server.crt /usr/local/share/ca-certificates/my-self-signed.crt sudo update-ca-certificates ``` Run Hitch: ``` /usr/local/sbin/hitch --test --config /etc/hitch/hitch.conf # validate conf first /usr/local/sbin/hitch --user hitch --group hitch --config /etc/hitch/hitch.conf --daemon ``` Test with curl: ``` curl -x https://www.yourdomain.com:8443 https://myip.ipip.net ``` # Test MTLS Create the 'client.ext' file with the following content: ``` # client.ext basicConstraints=CA:FALSE nsCertType=client nsComment='OpenSSL Generated Client Certificate' subjectKeyIdentifier=hash authorityKeyIdentifier=keyid,issuer:always keyUsage=critical,digitalSignature,keyEncipherment extendedKeyUsage=clientAuth ``` Create client certificate and private key: ``` openssl genrsa -out ca.key 2048 openssl req -new -x509 -days 3650 -key ca.key -out ca.crt openssl genrsa -out client.key 2048 openssl req -new -key client.key -out client.csr openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -extfile client.ext -out client.crt sudo cp ca.crt /etc/hitch/cert/ ``` Now add the following content in '/etc/hitch/hitch.conf' to enable MTLS on Hitch ``` # This forces clients to present a valid client certificate. client-verify = required # Client certificate authority file used to verify client certificates for mTLS. client-verify-ca = '/etc/hitch/cert/ca.crt' ``` Now run Hitch ``` /usr/local/sbin/hitch --test --config /etc/hitch/hitch.conf # validate conf first /usr/local/sbin/hitch --user hitch --group hitch --config /etc/hitch/hitch.conf --daemon ``` Verify MTLS setup with curl ``` MTLS TLS TLS curl -------> Hitch ------> Squid -----> Dst ``` ``` curl --cacert ca.crt --proxy-cert client.crt --proxy-key client.key -x https://www.yourdomain.com:8443 https://myip.ipip.net ``` > [!NOTE] > Should be noticed that currently only tls3 can be used in this case, https://github.com/curl/curl/issues/12286, so it's hard to verify from wireshark, since all certificate exchanges are encrypted after the TLS handshake. But we have Hitch configured to require MTLS, so if it succeeds, it should be using MTLS. # Enable Proxy Protocol The PROXY protocol was designed to help carry connection information from the source requesting side through a proxy server to the destination, especially when network address translation (NAT) is involved. It provides a way to preserve the original end-user IP addresses and TCP/UDP port numbers when traffic is routed through proxies and load balancers. The information is typically added to the header of the request forwarded to the destination server. There are two versions of the PROXY protocol: V1 and V2. - V1 is human-readable ASCII, V2 is binary. - V1 headers are simpler but more verbose, V2 headers are more flexible and can carry additional information using TLV (Type-Length-Value) vectors.. - V2 can support protocols other than TCP, e.g., UDP. It needs to be enabled at both Squid and Hitch. Add the following in '/etc/hitch/hitch.conf': ``` proxy-proxy = on ``` Change 'http_port 3128' in '/etc/squid/squid.conf': ``` http_port 3128 require-proxy-header proxy_protocol_access allow localnet # following one is needed when testing on the same machine acl localnet src 127.0.0.1 ``` Verify MTLS setup with curl ``` curl --cacert ca.crt --proxy-cert client.crt --proxy-key client.key -x https://www.yourdomain.com:8443 --haproxy-protocol https://myip.ipip.net ``` # Cert and Key Here's the step-by-step process for creating a set of certificates to be used for MTLS, including the CA certificate and key, server certificate and key, and client certificate and key. Please replace 'Your CA Name', 'yourdomain.com', and 'Your Server Name' with your actual CA, domain, and server names. ## Step 1: Generate the CA's Private Key ``` openssl genrsa -out ca.key 4096 ``` This creates a new 4096-bit RSA private key for your CA. The key will be stored in ca.key. ## Step 2: Generate the Self-Signed Root CA Certificate ``` openssl req -new -x509 -days 3650 -key ca.key -out ca.crt -subj '/CN=Your CA Name' ``` This command generates a new self-signed CA certificate based on the private key (ca.key) you created. The -days 3650 makes the certificate valid for 10 years. The certificate will be stored in ca.crt. ## Step 3: Generate the Server's Private Key ``` openssl genrsa -out server.key 2048 ``` Generate a new 2048-bit RSA private key for your server. The key will be stored in server.key. ## Step 4: Create a Certificate Signing Request (CSR) for the Server ``` openssl req -new -key server.key -out server.csr -subj '/CN=www.yourdomain.com' ``` This creates a new CSR (server.csr) for your server certificate using the server's private key (server.key). ## Step 5: Generate the Server Certificate Signed by Your CA ``` openssl x509 -req -days 365 -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt ``` This signs the server CSR with your CA key, resulting in a signed server certificate (server.crt) valid for 1 year. ## Step 6: Generate the Client's Private Key ``` openssl genrsa -out client.key 2048 ``` Generate a new 2048-bit RSA private key for the client. This key will be stored in client.key. ## Step 7: Create a CSR for the Client ``` openssl req -new -key client.key -out client.csr -subj '/CN=Your Client Name' ``` Create a CSR (client.csr) for the client using the newly created client key (client.key). ## Step 8: Generate the Client Certificate Signed by Your CA ``` openssl x509 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt ``` This signs the client CSR with your CA key, creating a signed client certificate (client.crt) valid for 1 year. This certificate is to be used for mutual TLS authentication by the client. ## Step 9 (optional): Verify the Certificates To verify the server and client certificates were signed by your CA: ``` openssl verify -CAfile ca.crt server.crt openssl verify -CAfile ca.crt client.crt ``` Both commands should return OK if everything went well. Now you have: - CA certificate (ca.crt) installed and trusted on the server and clients. - Server certificate (server.crt) and private key (server.key) are installed on the server. - Client certificate (client.crt) and private key (client.key) are installed on the client. Make sure you protect your private keys (ca.key, server.key, client.key) by setting appropriate file permissions. Using these instructions, your server and client can mutually authenticate themselves over a TLS connection, with your own CA acting as the trust anchor. This content is only supported in a Feishu Docs Reference https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt https://seriousben.com/posts/2020-02-exploring-the-proxy-protocol/。

Hands on experience on AI coding

Tue, 11 Feb 2025 01:22:37 +0000

I started using ChatGTP on writing some test code since last year, and recently I've heard people keep mentioning cursor, and used it to [make a app](https://www.reddit.com/r/cursor/comments/1h35uh6/nocode_developers_success_story_built_appstores_1/) in one hour and climbed to top 20 paid app in apple store, and he claim had no coding experience, maybe some just marketing, but still very impressive. I started to explorer the AI coding tools, like blot.new, windsurf, cursor, and finally became a windsurf paid user (tried for one month), and end up with building two tools, one Android app and one Web app. The Android app idea comes up TikTok Refugees' requirement, they switch to RedNote but there was language barriers there b/c most users are Chinese, so I came up with an idea of [screen translation](https://play.google.com/store/apps/details?id=com.lomoware.screen_translate&pli=1), I later thought I probably should do some research first before I did this, but since this starts with experimenting on AI coding tools, so I just launch and to see whether I can finish that within one week, the prototype is ready in 2 days, and I have no experience on flutter, I choose flutter on purpose b/c I'd like to see whether it's doable with zero experience, but I do have limited experience on Android/iOS development, but not expert. As for the prototype it just can capture the screen and do the translation, show overlay text, simple. However, it's far away from a product, there are a lot of improvements need to done to make it usable: 1. support multiple languages 2. detect scrolling so that it can translate the new content 3. support rotation 4. background color auto detection so that we can pick the overlay color 5. different modes (auto translation/manual translation/original text) Another tool I built with AI is https://insightly.top, an tool to 'Uncover Deep Insights from App Reviews', again, the tech stack is not something I familiar with, but I do have some basic knowledge on the frontend html+css+javascript. And similar time frame, around 1.5 week to go to production. I'm 100% that people with zero coding experience will not able to do this, some of those bugs are really tricky and I don't think AI can fix, and in the beginning when I use this tool, I thought it as reliable FSD, but it turns out to be a level2 auto pilot. I need to understand the code structure, the logic of how it works, and jump directly into some bug fixes, do the code review, but it DOES help improve the efficiency, and I don't think I can do the same thing within the same time frame. Some people mention AI coding are senior SDE now, I don't agree, it's not some equivalent, it's still a tool, without soul or creativity, still need human by its side to guide it, even though it can write code faster, and may have more knowledge than us. So basically those AI coding tools are just LLM + workflow integration, you just pay some convenient fees by saving the context switch effort. There are some prompts that helps to use the coding tools: - https://github.com/PatrickJS/awesome-cursorrules - https://cursor.directory/ - https://cursorlist.com/ They are helpful, but not enough, what I found that is sometimes we need play with multiple roles when working the project, you need to work on architect, frontend, backend, test, devops, and need to switch between different roles, and those prompts are just assuming you work on project with specific tech stack and that is not the future, in the future, AI will play different roles, either we have different agents with different roles to work together, or a full stack 10x AI agent can handle everything. The missing part is how to adapt the whole engineering process to AI era. > Software engineering is the process of designing, building, testing, and maintaining software applications. It's a branch of computer science that uses engineering principles and programming languages to create software solutions. What do software engineers do? > - Apply engineering principles to design software solutions > - Use programming languages to build software > - Understand algorithms, databases, and system design > - Solve problems > - Work in teams Anyway, I'm still exploring, and I won't go back to coding IDE without AI support, just like I won't go back to drive a car without [openpliot](https://github.com/commaai/openpilot), even though it's not perfect, it does helps, and it's the future.。

WebRTC和大语言模型

Wed, 08 Jan 2025 23:32:15 +0000

最近看到一家startup做的工作，将WebRTC和大语言多模态结合起来，做的一套实时音视频的SDK。

Mr. Resume: AI Resume writer

Wed, 08 Jan 2025 23:09:21 +0000

An prompt that helps on resume writing, thanks [Mr.-Ranedeer-AI-Tutor](https://github.com/JushBJJ/Mr.-Ranedeer-AI-Tutor), the prompt is available at https://github.com/MrResume/ResumeBoost/. The basic idea is to customize your resume according to a specific job description assisted with an AI resume writer, built this last year and made a landing page for that, there could be some potential to build a business around this idea. Just watched two YouTube videos related to this, very inspiring: `Gmeek-html` `Gmeek-html` # ResumeBoost Resume Boost with ChatGPT / Claude # Usage Copy paste [ResumeBoost.txt](ResumeBoost.txt) into https://chat.openai.com or https://claude.ai. ``` /config: Configure your resume preferences, such as experience level, industry, tone style, resume length, and language. /start: Begin the process of collecting information for your resume. /done: Generate your resume based on the information you've provided. /analyse: Analyze your resume and rate it based on a job description. /continue: If you have more work experiences, education, certifications, or languages to add after generating your resume. /language [lang]: Change the language of the assistant to the specified language. /example: Get an example of how your resume might look for a specific job description. ```。

OpenAI coding question: OpenSheet

Wed, 08 Jan 2025 22:16:36 +0000

# Question Today we’re going to build a basic spreadsheet application like Google sheets or Excel but much simpler. Our spreadsheet, let’s call it OpenSheet, will only support cells which hold either integers or formulas that sum two cells. You are tasked with writing a program that handles this functionality for OpenSheet. You can make any decisions you want regarding how this program is organized, but there must be some sort of setter/getter methods that can be called by the application for any given cell. All inputs will be strings. For setting you can expect two inputs: the cell location and the cell value. Example of how your setter could look ``` set_cell('C1', '45') set_cell('B1', '10') set_cell('A1', '=C1+B1') ``` For getting you will be provided one input that is the cell location. Example of how your getter could look ``` get_cell('A1') # should return 55 in this case ``` Assumptions: - In memory storage - All cell location inputs will be well formed (no need to validate in code) - All cell value inputs will be well formed (no need to validate in code) - Cells value inputs are either a summation of two other cells or an int - Empty cells are treated as zero when accessed # Solutions One of the solutions found on the internet is as below, so basically 1. store different types of cells separately 2. replace the formula cell with valued cell for evaluation when getting the cell ```python import re class Spreadsheet: def __init__(self): self.cells = {} self.formula_cells = {} def set_cell(self, cell, value): self._clear_all(cell) if value.startswith('='): self.formula_cells[cell] = value else: self.cells[cell] = value def _evaluate_formula(self, value): formula = value for cell in re.findall(r'[A-Z][0-9]+', value): formula = formula.replace(cell, self.cells[cell]) return eval(formula[1:]) def get_cell(self, cell): if cell in self.cells: return int(self.cells[cell]) elif cell in self.formula_cells: value = self.formula_cells[cell] return self._evaluate_formula(value) return 0 def _clear_all(self, cell): if cell in self.cells: self.cells.pop(cell) if cell in self.formula_cells: self.formula_cells.pop(cell) ss = Spreadsheet() ss.set_cell('A1', '13') ss.set_cell('A2', '14') assert ss.get_cell('A1') == 13 ss.set_cell('A3', '=A1+A2') assert ss.get_cell('A3') == 27 ``` > [!WARNING] > The problem with this solution is that it doesn’t consider the case that the items in formula cell could be another formula cell, like: ```python ss.set_cell('A1', '13') ss.set_cell('A2', '14') ss.set_cell('A3', '=A1+A2') # will raise KeyError: 'A3' ss.set_cell('A4', '=A3+A1') ``` And the fix is to call “_evaluate_formula” recursively ```python def _evaluate_formula(self, value): formula = value for cell in re.findall(r'[A-Z][0-9]+', value): cell_value = 0 if cell in self.cells: cell_value = self.cells[cell] elif cell in self.formula_cells: cell_value = self._evaluate_formula(self.formula_cells[cell]) formula = formula.replace(cell, str(cell_value)) return eval(formula[1:]) ``` > [!WARNING] > Another corner case is “circular dependency” ```python ss.set_cell('A1', '13') ss.set_cell('A2', '14') ss.set_cell('A3', '=A1+A2') # will raise RecursionError ss.set_cell('B1', '=A1+C1') ss.set_cell('C1', '=B1+A2') ss.set_cell('B2', '=B2+A3') ``` To fix this, record the visited cell when evaluating each time and raise an error if “circular dependency” is found: ```python def __init__(self): ... self.visited = set() def _evaluate_formula(self, value): formula = value for cell in re.findall(r'[A-Z][0-9]+', value): cell_value = 0 if cell in self.cells: cell_value = self.cells[cell] elif cell in self.formula_cells: if cell in self.visited: raise ValueError('Circular Dependency!') self.visited.add(cell) cell_value = self._evaluate_formula(self.formula_cells[cell]) formula = formula.replace(cell, str(cell_value)) return eval(formula[1:]) def get_cell(self, cell): self.visited.clear() if cell in self.cells: return int(self.cells[cell]) elif cell in self.formula_cells: value = self.formula_cells[cell] return self._evaluate_formula(value) return 0 ``` > [!TIP] > Another enhancement could be done to improve performance is by catching the value instead of recalculated every time. This adds some complexity b/c once one cell change, all the value of cells depends on it will change as well, and the cached value will be invalid anymore, to achieve this we need to maintain the dependencies, given a cell knows which cells deps on it. Here is another solution, deps is used to maintain the cell sets that rely on specific cell, so that once we update this specific cell, we can clear the cache for those cells rely on it. Another thing need to notice is that when removing the cache, there could be chance that it has “circular dependency” as well, so should check that too. ```python from collections import defaultdict class Cell: def __init__(self, key, expr): self.key = key self.value = self.left_key = self.right_key = None self._parse(expr) def _parse(self, expr): if expr.isdigit(): self.value = int(expr) else: self.left_key, self.right_key = expr[1:].split('+') class Spreadsheet: def __init__(self): self.key_to_cell = {} self.visited = set() self.cache = {} self.deps = defaultdict(set) def set_cell(self, key, expr): if key in self.key_to_cell: self._clean_cache(key) self._remove_deps(self.key_to_cell[key]) self.key_to_cell.pop(key) cell = Cell(key, expr) self._add_deps(cell) self.key_to_cell[key] = cell def _add_deps(self, cell): if cell.value is None: self.deps[cell.left_key].add(cell.key) self.deps[cell.right_key].add(cell.key) def _remove_deps(self, cell): if cell.value is None: self.deps[cell.left_key].remove(cell.key) self.deps[cell.right_key].remove(cell.key) def _clean_cache(self, key): self.visited.clear() self._do_clean_cache(key) def _do_clean_cache(self, key): if key in self.cache: self.cache.pop(key) if key not in self.deps: return if key in self.visited: return for k in self.deps[key]: self.visited.add(k) self._do_clean_cache(k) def get_cell(self, key): self.visited.clear() try: return self._evaluate(key) except ValueError as e: print(f'error fetching value for {key}: {e}') return 0 def _evaluate(self, key): if key not in self.key_to_cell: return 0 if key in self.cache: return self.cache[key] cell = self.key_to_cell[key] if cell.value is not None: return cell.value if cell.key in self.visited: raise ValueError('Circular Dependencies!') return 0 self.visited.add(cell.key) res = self._evaluate(cell.left_key) + self._evaluate(cell.right_key) self.cache[key] = res return res ss = Spreadsheet() ss.set_cell('A1', '13') ss.set_cell('A2', '14') assert ss.get_cell('A1') == 13 ss.set_cell('A3', '=A1+A2') assert ss.get_cell('A3') == 27 ss.set_cell('A4', '=A3+A1') assert ss.get_cell('A4') == 40 ss.set_cell('B1', '=A1+C1') ss.set_cell('C1', '=B1+A2') ss.get_cell('B1') ss.set_cell('A4', '=A2+A1') assert ss.get_cell('A4') == 27 ss.set_cell('A2', '15') assert ss.get_cell('A4') == 28 ```。

伪代码式提示：让AI高效听懂你的需求！

Wed, 08 Jan 2025 21:46:28 +0000

# 什么是伪代码式提示？不是写代码！而是用逻辑清晰、结构化的方式告诉AI你想要什么，让AI更懂你，少踩坑，多省心！ # 为什么要用伪代码式提示？ 1️⃣ 逻辑清晰：明确每一步要干啥，AI按流程来，结果不乱套！ 2️⃣ 内容完整：不会忘东忘西，细节都能照顾到。

BigBlueButton的开源商业模式

Wed, 08 Jan 2025 21:37:44 +0000

Bigbluebutton CEO Fred Dixon曾说过：”创建一个成功的公司不容易，但同时创建一个成功的公司，成功的开源项目和成功的生态系统就是难上加难了。

Piem: Network Emulator based on Raspberry Pi

Wed, 08 Jan 2025 21:33:45 +0000

# Basic setup You can use [Raspberry Pi 4](https://www.amazon.com/GeeekPi-Raspberry-4GB-Starter-Kit/dp/B0B3M2HKN6/ref=sr_1_12_sspa?crid=2L2JEUJ1460WB&dib=eyJ2IjoiMSJ9.dm2FB5zZz0j71xQemzz3eGcfMj6yAgNj0EPv-ZREy2PWNx49GJkdHYAzErmjO1rWARxfo12XUNdoTscBLFfbLbRIYyt2O44eh6kBKePKgZHoWzqG1x61SEy_n7vicrrWGgnVkLoWBoS1rY0LU453Xc8SXZtyYgzT9ZGvLWKovR1k4qtC_FNH2zz6wdvKDwm1cEi8riFowfxHNBwFO9r6uRsuyK5wCfuxhzNEeM3-ugw.eOUzPFP7-jr6HburUXKHBVMMswaal6qpIhVdoWuOVjI&dib_tag=se&keywords=raspberry+pi+starter+kit&qid=1726551003&sprefix=raspberry+pi+starter+ki%2Caps%2C176&sr=8-12-spons&sp_csd=d2lkZ2V0TmFtZT1zcF9tdGY&psc=1) (or Raspberry Pi 3B) as a network emulator. Raspberry Pi can be used to set up a bridged network, either using ethernet or WiFi. You can [download](https://www.dropbox.com/scl/fi/e02utgcmfw5opfymdq9h7/image_2023-05-10-piem-lite.zip?rlkey=l8b32tgl41zoqa6nupxq2y41r&st=ad66ldbu&dl=0) the prebuild image, and then download [Etcher](https://etcher.io/) and install the image on the micro sdcard. Once done, connect keyboard monitor, and ethernet, boot and login with username “pi” and password “raspberry”, then execute the following command to enable WiFi network: ``` rfkill unblock all ``` Then reboot, login again and make sure the network is working: using ping command to make sure the internet access is working Use cell phone to scan the wifi network and find ssid “piem”, and login with password “piemulator”, and make sure the internet access is working If not working, try run `sudo systemctl restart networking.service` and wait a few minutes Then you can test network impairment: Run speed test on Raspberry Pi ``` sudo apt install speedtest-cli speedtest-cli Retrieving speedtest.net configuration... Testing from Comcast Cable (98.37.129.143)... Retrieving speedtest.net server list... Selecting best server based on ping... Hosted by Jefferson Union High School District (Daly City, CA) [62.00 km]: 16.523 ms Testing download speed................................................................................ Download: 483.86 Mbit/s Testing upload speed...................................................................................................... Upload: 112.79 Mbit/s ``` 2. Run speed test on cell phone connected with WiFi “piem”. (The speed test may not be as good as you run it on Raspberry Pi, most likely due to interference, default the channel is set to 1) 3. Test rate limit, set downlink bandwidth to 3mbps, “192.168.1.185” is the IP address of the cellphone ``` sudo emulator.py add -b 3000 -f 192.168.1.185 -c downlink ``` 4. Then run speed test on cell phone again to validate the settings 5. Remove the rate limit and test it again ``` sudo emulator.py remove -f 192.168.1.185 -c downlink ``` 6. Run sudo emulator.py -h for help You can refer https://github.com/cellsim/piem for more details # Performance tune Modify “/etc/hostapd/hostapd.conf”, change/add the following settings: 1. Use 5GHz network 2. Search for the channel with the least interferences 3. Enable 802.11n support 4. Enable 802.11ac support 5. Enable QoS support 6. Enable HT40 with short guard intervals ``` hw_mode=a channel=0 ieee80211n=1 ieee80211ac=1 wmm_enabled=1 ht_capab=[HT40+][SHORT-GI-40] ``` Then restart hostapd: ``` sudo systemctl restart networking.service ``` There are chances that channel = 0 doesn't work (at least for me), and you won't be able to find 'piem' network anymore. You can enable logging by adding the following line in '/etc/default/hostapd': ``` DAEMON_OPTS='-dd -t -f /tmp/hostapd.log' ``` Restart hostapd and make sure it runs successfully by checking systemctl status hostapd.service, otherwise check error log. If channel auto selection doesn’t work, change that to channel = 36 or other Non-DFS Channels like: 40, 44, 48, 149, 153, 157, 161, 165. Use the following command on Raspberry Pi to check the channel in use nearby: ``` sudo iwlist wlan0 scan | grep 'Channel' ``` After making those changes, you should be able to see the performance boost (my test shows 3x improvement, around 60mbps ~ 80mbps).。

TCP Cubic Congestion Control Paper and Linux Implementation

Wed, 08 Jan 2025 05:34:54 +0000

# Paper use a cubic function to replace the linear window growth to improve performance in high BDP network, also achieved RTT fairness. tcp-cubic is succssor of tcp-bic, tcp-bic use binary search to find the avialble bandwidth when congestion happens, and MAY enter an lieaner increasing state (Additive Increase)if it dropped too low, if it passed the maximum window, it goes into max probing state and will do the reverse and grow slowly initially and switch to additive increase if it can't find new maximum window nearby. The problem with tcp-bic is that the growth function is too aggressive for short RTT or low speed network, and different phases (additive increase, binary search, max probing) add complexity in implememting and analyzing. tcp-cubic is replacing the phase functions with a unified one function, which is a cubic function of the elapsed time from the last congestion event. for the tcp-cubic function, we can start with `w=t^^3`, which gives the initial shape center at (0, 0), then we can move the center towards the right to K, which gives `w = (t-K)^^3`, and then move the center upwards to Wmax, then we have `w = (t-K)^^3 + Wmax`, and we can tune the shape via parameter C, the larger C, more aggressive on the steps, now we have **equation (1)** `w = C*(t-K)^^3 + Wmax`. Say if now we have congestion and the congestion window drops from `Wmax` to `Wmax*(1 - beta)`, beta is the window decrease factor. And we assume this point is `(0, Wmax*(1 - beta)`, then K is actually the time it takes to ramp up back to Wmax. and we can get the **equation (2)**: ``` cwnd = Wmax - Wmax * beta = C * (0 - K)^^3 + Wmax => Wmax * beta = C * K^^3 => K = (Wmax * beta / C)^^(1/3) ``` That is how we get the eqaution (1) and (2) in the paper. In each RTT period, we calcuate curent Window size using equation (1), `W_curr = W(t + RTT)`. The current window might be to small compared with traditional TCP, and might cause friendly issue(actually no wrose than traditional TCP in anyway), so it will give an estimation of the traditional TCP congestion window after time t, and if the window calcualated using cubic function is smaller than that, it will use the traditional TCP congestion window. To make it fairness with traditional TCP, we should get similar average window size. The average window size of AIMD additive increase (alpha) and multplicative decrease (beta) is give as `sqrt[alpha/2 * (2-beta)/beta / p] / RTT`, for tcp alpha = 1 and beta = 0.5, so we have average window size for traditional TCP `sqrt(3/2/p) / RTT`. Since now in use different beta (0.2) for cubic, to get similiar average window size, the alpha should be `3 * beta / (2 - beta)`, and we can caculate the window size if using AIMD with alpha and beta, given the elapsed time t since last decrease. ``` Wtcp = Wmax * (1-beta) + alpha * t / RTT = Wmax * (1-beta) + 3 * beta / (2-beta) * t / RTT ``` fast convergence is used to let new flow to get its fair share more quickly, the idea is that when loss event occurrs, if the Wmax keeps dropping (less than before), it will reduce more to apply a factor `1 - beta/2` (that is * 0.9), so that the flow have the plateau earlier to give other flows more chance to catch up. # Linux implementation At a first glance, the Linux implementation looks more complicate than the paper, the reason is that for kernel peformance, for example it convert all the float number operation into integer by scaling, and also the congestion avoidance implementation is actually increase the congestion window by 1 every N ACKs, it won't be able to increase by a float number calculated using the equation (1), we have to calculate a N to approximate that to achieve similar effect, and need to consider the effect of delay-ack. Another confusing thing is that the beta in linux implementation is actually 1-beta in the paper. The code also include the hystart slow start algorithm which is not in the cubic paper. First, let's see how the code use the equations above, the first thing is when there is a packet drop, it will update `ssthresh` and `last_max_cwnd`, beta is 717 and BICTCP_BETA_SCALE is 1024, beta/BICTCP_BETA_SCALE = 0.7, so the `snd_ssthresh` will drop to `0.7 * snd_cwnd`. And `snd_cwnd` will be updated based on the state of TCP (todo add more details, see tcp_input.c) If no fast convergence, `last_max_cwnd` will set to the current `snd_cwnd`, otherwise it will apply another factor: `(BICTCP_BETA_SCALE + beta) / (2 * BICTCP_BETA_SCALE) = (1+beta_before_scaled)/2 = (1+1-beta_in_paper)/2 = (2 - beta_in_paper)/2 = (1 - beta_in_paper/2` ``` static u32 bictcp_recalc_ssthresh(struct sock *sk) { const struct tcp_sock *tp = tcp_sk(sk); struct bictcp *ca = inet_csk_ca(sk); ca->epoch_start = 0; /* end of epoch */ /* Wmax and fast convergence */ if (tp->snd_cwnd < ca->last_max_cwnd && fast_convergence) ca->last_max_cwnd = (tp->snd_cwnd * (BICTCP_BETA_SCALE + beta)) / (2 * BICTCP_BETA_SCALE); else ca->last_max_cwnd = tp->snd_cwnd; ca->loss_cwnd = tp->snd_cwnd; return max((tp->snd_cwnd * beta) / BICTCP_BETA_SCALE, 2U); } ``` When get ack, it will calculate a moving average of delayed acked packet count `delayed_ack`, `ACK_RATIO_SHIFT`(default to 4) is the moving average factor and `cnt` is the delayed acked packet count in current ACK. `delayed_ack` will be used later to update congestion window, in the paper it assumes that we update congestion window on each ACK per RTT, and delayed ack might gives longer interval per ACK, and we need compensate for that. The following code is actuall doing the math `ratio = (15*ratio + sample) / 16`, `delayed_ack`(ratio) is initilized to be 16 times larger `ca->delayed_ack = 2 << ACK_RATIO_SHIFT;`, if the last value is X without scaling by 16 times (`ratio = 16 * X`), then `15 * ratio / 16 + cnt = 15 * 16 * X / 16 + cnt = 15 * X + cnt`, `15 * ratio / 16 + cnt` is what used in the code below, and if we check the above equation in reverse direction: moving average of the raw value X without scaling is `(15 * X + cnt) / 16 = (15 * 16 * X / 16 + cnt) / 16 = (15 * ratio / 16 + cnt) / 16`, that is we can calculate moving average X using the scaled value ratio, and we need `* 16` to convert the raw value to `ratio` for next iteration, and store that in `delayed_ack`, so the `delayed_ack` is actually the moving avarege of scaled value. ``` static void bictcp_acked(struct sock *sk, u32 cnt, s32 rtt_us) { ...... u32 ratio = ca->delayed_ack; ratio -= ca->delayed_ack >> ACK_RATIO_SHIFT; ratio += cnt; ca->delayed_ack = min(ratio, ACK_RATIO_LIMIT); ...... } ``` If the current `snd_cwnd` is no larger than `snd_ssthresh` it will enter into slow start, can either use the traditional slow start or hystart based on the settings, otherwise, it will enter into congestion avoidance, this is what cubic does most of its job: If current `cwnd` is larger than `last_max_cwnd`, will reset the origin point to `cwnd`, and reset `bic_K` to zero, this is the max probing state. Else it's in steady state, and will compute new K, the estimate time needed to reach `last_max_cwnd`, we can rewrite equation 2 `cwnd = Wmax - Wmax * beta = C * (0 - K)^^3 + Wmax => K = [(Wmax - cwnd) / C]^^(1/3)`, this is the equation used in the code. ``` if (ca->last_max_cwnd <= cwnd) { ca->bic_K = 0; ca->bic_origin_point = cwnd; } else { /* Compute new K based on * (wmax-cwnd) * (srtt>>3 / HZ) / c * 2^(3*bictcp_HZ) */ ca->bic_K = cubic_root(cube_factor * (ca->last_max_cwnd - cwnd)); ca->bic_origin_point = ca->last_max_cwnd; } ``` should be notice that the unit of bic_K (same for the elapsed time since epoch) is not ms nor jiffies(HZ), it's using `BHZ = HZ*(2^^BICTCP_HZ) = HZ*(2^^10) = HZ*1024`, to avoid overflow when doing the math. The C used is actually `bic_scale * 10 / 1024 = 41 * 10 / 1024 = 0.4`, and in order to convert the time to units of BHZ, `bic_K (HZ) = cubic_root(1/C * (last_max_cwnd - cwnd)) => bic_K (BHz) * 2^^BICTCP_HZ = 2^^BICTCP_HZ * cubic_root(1/C * (last_max_cwnd - cwnd)) = cubic_root((2^^BICTCP_HZ)^^3 * 1/C * (last_max_cwnd - cwnd)) = cubic_root(2^^(3*BICTCP_HZ) * 1/C * (last_max_cwnd - cwnd)) = cubic_root(2^^(3*BICTCP_HZ) * 2^^10/(bic_scale*10) * (last_max_cwnd - cwnd))` `2^^(3*BICTCP_HZ) * 2^^10/(bic_scale*10) = 2^^(3*BICTCP_HZ+10)/(bic_scale*10) = 1ull << (10+3*BICTCP_HZ)/(bic_scale*10)`, this is how `cube_factor` is caculated. ``` cube_factor = 1ull << (10+3*BICTCP_HZ); do_div(cube_factor, bic_scale * 10); ``` Then we will caculate the elapsed time plus one RTT in unit BHZ, and then caculate the target congestion window, for calculating delta of congestion window, `delta = (cube_rtt_scale * offs * offs * offs) >> (10+3*BICTCP_HZ);`, since the `offset` is in BHZ, to convert it back to HZ, we need `/ (2^^BICTCP_HZ)`, that is `>>BICTCP_HZ`, and we have multiplied 3 offset, so it's converting 'BHZ * BHZ * BHZ' back to 'HZ * HZ * HZ', that is `/(2^^BICTCP_HZ)^^3 = /(2^^(3*BICTCP_HZ) = >>(3*BICTCP_HZ)`, and `C = cube_rtt_scale/1024 = cube_rtt_scale >> 10`. so `delta = C * offs * offs * offs >> (3+BICTCP_HZ) = cube_rtt_scale * offs * offs * offs >> (10+3*BICTCP_HZ)` ``` t = ((tcp_time_stamp + msecs_to_jiffies(ca->delay_min>>3) - ca->epoch_start) << BICTCP_HZ) / HZ; if (t < ca->bic_K) /* t - K */ offs = ca->bic_K - t; else offs = t - ca->bic_K; /* c/rtt * (t-K)^3 */ delta = (cube_rtt_scale * offs * offs * offs) >> (10+3*BICTCP_HZ); if (t < ca->bic_K) /* below origin*/ bic_target = ca->bic_origin_point - delta; else /* above origin*/ bic_target = ca->bic_origin_point + delta ``` Once we have `bic_target`, we can calculate the congestion window increment for each RTT, `tcp_cong_avoid_ai` is called for each ACK, and it will increase `snd_cwnd` by 1 on every `w` ACKs (that is increase by `1/w` on each ACK), for TCP reno `w` is the congestion window, that is increment by 1 for each RTT (receive `w` ACKs when `w` packets send out in one RTT). ``` /* In theory this is tp->snd_cwnd += 1 / tp->snd_cwnd (or alternative w) */ void tcp_cong_avoid_ai(struct tcp_sock *tp, u32 w) { if (tp->snd_cwnd_cnt >= w) { if (tp->snd_cwnd < tp->snd_cwnd_clamp) tp->snd_cwnd++; tp->snd_cwnd_cnt = 0; } else { tp->snd_cwnd_cnt++; } } ``` For cubic, we are expecting to increase the congestion window by `bic_target - cwnd` in the next round trip time, since we will send out `cwnd` packets in one RTT, if we don't consider delayed ack and assume ack per packet, then it's for each ACK the congestion window increases by `(bic_target - cwnd)/cwnd`, then `(bic_target - cwnd)/cwnd = 1/w => w = cwnd/ (bic_target - cwnd)`, which is exactly what we have for `ca->cnt` when `bic_target > cwnd`, if it's smaller, then the increment will be very small, `1/(100 * cwnd)` on each ACK, that is `1/100` on each RTT if no delay ACK, this is just a safety guard. ``` /* cubic function - calc bictcp_cnt*/ if (bic_target > cwnd) { ca->cnt = cwnd / (bic_target - cwnd); } else { ca->cnt = 100 * cwnd; /* very small increment*/ } ... tcp_cong_avoid_ai(tp, ca->cnt); ``` And if we consider the delay ack, we should compensate for that b/c we may get less Ack when using delay ack, which means for each ACK we should increase more, that said if no delay ACK, we increase `1/ca->cnt` per ACK (w = ca->cnt), then with delay ACK, assume we have pkts_per_ack, we should increase `pkts_per_ack/ca->cnt` per ACK (w = ca->cnt/pktsPerAck . `delayed_ack` is a scaled value, then `pkts_per_ack = delayed_ack/16` ``` ca->cnt = (ca->cnt << ACK_RATIO_SHIFT) / ca->delayed_ack; ``` To avoid the case cubic is slower than TCP, it will estimated the congestion window of traditional TCP. In the paper we have the increament factor `alpha = 3 * beta_in_paper / (2-beta_in_paper)`, since in the code the beta is actual `1-beta_in_paper`, then `alpha = 3 * (1 - beta) / (1 + beta)`. in the code, `beta_scale = 8 / alpha`, 8 is just another scaling factor which ensure beta_scale is integer, and for later use it will apply `>>3` which give `1 / alpha`. ``` beta_scale = 8*(BICTCP_BETA_SCALE+beta)/ 3 / (BICTCP_BETA_SCALE - beta); ``` When `tcp_friendliness` enabled, to simulate traditional TCP, the congestion window increase per RTT is `alpha`, so for each ACK, the increment `1/w` is `alpha/cwnd = 8/(beta_scale * cwnd)`, so we have `w = beta_scale * cwnd / 8 = beta_scale * cwnd >> 3`. That is every `w` ACK packets the congestion window increment by 1, give current has `ca->ack_cnt` acks in total, we can estimate `tcp_cwnd`. once we have `tcp_cwnd`, if it's larger than current `cwnd`, we should increase cwnd by `delta = tcp_cwnd - cwnd`, and we use the max between `bic_target` and `tcp_cwnd`, the code is using smaller cnt but it's actually the same. ``` /* TCP Friendly */ if (tcp_friendliness) { u32 scale = beta_scale; delta = (cwnd * scale) >> 3; while (ca->ack_cnt > delta) { /* update tcp cwnd */ ca->ack_cnt -= delta; ca->tcp_cwnd++; } if (ca->tcp_cwnd > cwnd){ /* if bic is slower than tcp */ delta = ca->tcp_cwnd - cwnd; max_cnt = cwnd / delta; if (ca->cnt > max_cnt) ca->cnt = max_cnt; } } ``` # Hystart Linux implementation also use hystart slow start by default, hystart slow start is to exit early to avoid too many packet loss caused by slow start phase, and it utilize the packet train and delay to give hint when to stop slow start. Since for most TCP it's window based and data are send out in a burst in one congestion window, which means we can use those packets as packet train, suppose we send out N packets in the train, the time gap between the 1st and Nst packet is delta(N), then the bandwidth estimation will be `bw = (N-1) * packet_length / delta(N)`, the network pipe capcaity (without buffer) is `K = bw * one_way_delay_min`, the data sent in a cwnd should be no larger than the pipe capacity. ``` (N-1) * packet_length <= bw * one_way_delay_min = (N-1) * packet_length / delta(N) * one_way_delay_min => delta(N) <= one_way_delay_min ``` That means we can measure the time gap of the train to know whether we are exceed the available bandwidth and enter into congestion avoidance. Since it's not easy to measure one way delay, half of RTT is used, and to avoid modification on both sides, ACK gap between Nst and 1st packet is used. Using RTT/2 as one way delay estimation won't make things worse, exit too early will cause under untilization, `beta * (bw * one_way_delay_min)` can be used as the lowerbound for safety exit, for standard TCP, beta is 0.5, other variances has beta larger than 0.5; suppose we have forward and backward delay `a` and `b`, if we use `RTT/2 = (a+b)/2` as one way delay, the BDP estimation `K' = bw * (a+b)/2` while `K = bw * a`, `K'/K = (a + b) / 2 / a = 1/2 + b/2/a >= 1/2`, so we have `K' >= 0.5 * K`, that is to say if we use RTT/2 as estimation, the BDP estimations is no less than half of the real BDP, so we won't exit slow start before we reach 0.5 * BDP (threshold of the standard TCP slow start). And if it congestin window goes beyond `K + S`(S is network buffer size), which is the upper bound of safety exit, packet will be dropped, so we have `K'/K = 1/2 + b/2/a <= (K + S)/K => 1/2 + b/2/a <= 1 + S/K`, if `S=K`, then `1/2 + b/2/a <= 2 => b/a <= 3 => b <= 3a`, and there are less than 5% cases that has reverse path delay larger than forward path, which means in this case, the slow start will fallback to traditional slow start which overshoots and cause packet loss. Using Ack may give larger time gap, which gives lower bandwidth estimation, and cause conservative behavior to exit slow start earlier. Delay Ack also affect the accuracy, so if there is significantly delay in the last ACK, that sample is filter out and mixed with next train. And there may be cases that minimum RTT is not available, for example when multiple flows are competing, the idea is use delay increasing as an exit indicator, it measure the first a few packets of the train, and calculate the average RTT for that train, and compare the trian K and train K-1, if RTT(K) > RTT(K-1) + delta, then exit slow start. The Linux implementation is slightly different and simpler than that in the paper. Hystart will be triggered when the cwnd is larger than `hystart_low_window` (default to 16) ``` /* hystart triggers when cwnd is larger than some threshold */ if (hystart && tp->snd_cwnd <= tp->snd_ssthresh && tp->snd_cwnd >= hystart_low_window) hystart_update(sk, delay); ``` It will keep track to minimum delay(`delay_min`) in the tcp session so far. On each ACK, it will check if the time since last ACK is less than the threshold `hystart_ack_delta` to filter the invalid sample, if it's invalid, then in this round, it will not do ack train detection, and if the sample is valid, check if it goes beyond minRTT/2 (`delay_min` is scaled by 8, so `>>4` is actually minRTT/2), if so it will detect as `HYSTART_ACK_TRAIN` happens. It also track minimum delay(`curr_delay`) among the first `HYSTART_MIN_SAMPLES` samples in each round (send cwnd packets in the burst train). It `curr_delay` is larger than `delay_min` more than a threshold value, which is `delay_min/2`, clamp to `[4, 16] ms`, then it will detect as `HYSTART_DELAY`. Either one detected will cause the slow start exit by setting the `ssthresh` to current congeston window `snd_cwnd`. ``` static void hystart_update(struct sock *sk, u32 delay) { struct tcp_sock *tp = tcp_sk(sk); struct bictcp *ca = inet_csk_ca(sk); if (!(ca->found & hystart_detect)) { u32 now = bictcp_clock(); /* first detection parameter - ack-train detection */ if ((s32)(now - ca->last_ack) <= hystart_ack_delta) { ca->last_ack = now; if ((s32)(now - ca->round_start) > ca->delay_min >> 4) ca->found |= HYSTART_ACK_TRAIN; } /* obtain the minimum delay of more than sampling packets */ if (ca->sample_cnt < HYSTART_MIN_SAMPLES) { if (ca->curr_rtt == 0 || ca->curr_rtt > delay) ca->curr_rtt = delay; ca->sample_cnt++; } else { if (ca->curr_rtt > ca->delay_min + HYSTART_DELAY_THRESH(ca->delay_min>>4)) ca->found |= HYSTART_DELAY; } /* * Either one of two conditions are met, * we exit from slow start immediately. */ if (ca->found & hystart_detect) tp->snd_ssthresh = tp->snd_cwnd; } } ``` # Tuning parameters Linux cubic implementation has some parameters which can be used to tune the algorithm, by set different values for files (filename is the same with parameter name) in `/sys/module/tcp_cubic/parameters/` - fast_convergence: default is on. enable fast convergence will degrade more when consecutive downgrade, this is used for let new comers to get fair share faster. - beta: beta is used for multiplicative decrease, larger value will decrease less, default is 0.7, that is 0.7 * cwnd when loss happens. beta will also affect the fairness with standard TCP. beta is scaled by 1024. - initial_ssthresh: use to set the initial ssthresh value, only used at the beginning whe hystart not enabled. - bic_scale: bi_scale is used to tune the cubic funtion curve (C in the equation 1), large beta will increase more aggressively, default is 0.4. bic_scale also affect teh fairness with standard TCP. bic_scale is scaled by 1024. - tcp_friendliness: used to ensure in any case cubic is no worse than standard TCP. default is on. - hystart: enable hystart, which exit slow start earlier based on delay to avoid too much packet loss. - hystart_detect: enable which methods are used for hystart detection, default is enable both packet train and delay increase. - hystart_low_window: when cwnd is larger than hystart_low_window, will start hystart slow start. default is 16. - hystart_ack_delta: threshold value to filter delay ACK, default is 2. ``` module_param(fast_convergence, int, 0644); MODULE_PARM_DESC(fast_convergence, 'turn on/off fast convergence'); module_param(beta, int, 0644); MODULE_PARM_DESC(beta, 'beta for multiplicative increase'); module_param(initial_ssthresh, int, 0644); MODULE_PARM_DESC(initial_ssthresh, 'initial value of slow start threshold'); module_param(bic_scale, int, 0444); MODULE_PARM_DESC(bic_scale, 'scale (scaled by 1024) value for bic function (bic_scale/1024)'); module_param(tcp_friendliness, int, 0644); MODULE_PARM_DESC(tcp_friendliness, 'turn on/off tcp friendliness'); module_param(hystart, int, 0644); MODULE_PARM_DESC(hystart, 'turn on/off hybrid slow start algorithm'); module_param(hystart_detect, int, 0644); MODULE_PARM_DESC(hystart_detect, 'hyrbrid slow start detection mechanisms' ' 1: packet-train 2: delay 3: both packet-train and delay'); module_param(hystart_low_window, int, 0644); MODULE_PARM_DESC(hystart_low_window, 'lower bound cwnd for hybrid slow start'); module_param(hystart_ack_delta, int, 0644); MODULE_PARM_DESC(hystart_ack_delta, 'spacing between ack's indicating train (msecs)'); ```。

吉他和弦基础理论

Wed, 08 Jan 2025 00:52:08 +0000

## 音符音乐有12个音符(Note)，每个音符之间隔一个半音，比如B和C之间就隔一个半音(H)，A和B之间隔一个全音(W, 即两个半音): A | A♯/B♭ | B | C | C♯/D♭ | D | D♯/E♭ | E | F | F♯/G♭ | G | G♯/A♭ :---:|:-------------------:|:---:|:---:|:-------------------:|:---:|:------------------:|:---:|:---:|:-------------------:|:---:|:------------------: 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 音符在吉他上的位置，从左到右对应0-12个音格(Fred)，从上到下对应1-6弦： 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 :---:|:-------------------:|:---:|:---:|:-------------------:|:---:|:------------------:|:---:|:---:|:-------------------:|:---:|:------------------: E | F | F♯/G♭ | G | G♯/A♭ | A | A♯/B♭ | B | C | C♯/D♭ | D | D♯/E♭ B | C | C♯/D♭ | D | D♯/E♭ | E | F | F♯/G♭ | G | G♯/A♭ | A | A♯/B♭ G | G♯/A♭ | A | A♯/B♭ | B | C | C♯/D♭ | D | D♯/E♭ | E | F | F♯/G♭ D | D♯/E♭ | E | F | F♯/G♭ | G | G♯/A♭ | A | A♯/B♭ | B | C | C♯/D♭ A | A♯/B♭ | B | C | C♯/D♭ | D | D♯/E♭ | E | F | F♯/G♭ | G | G♯/A♭ E | F | F♯/G♭ | G | G♯/A♭ | A | A♯/B♭ | B | C | C♯/D♭ | D | D♯/E♭ ## 音阶(Scale) 常见的有大调音阶和小调音阶，它们的区别在于全音(W)和半音(H)的排列顺序不同，而这种不同的顺序会带来不同的风格。

The Pragmatic Programmer读后感

Wed, 08 Jan 2025 00:11:47 +0000

这本书的副标题是From Journeyman to Master，中文翻译过来是”从小工到专家”，工作了近十年，回过头来再看这本书，发现这本书虽然从出版到现在已经近20年了，正如作者在09年的再版序言里说的:”Things Really Haven’t Changed That Much”，又一个十年过去了，书中的最佳实践放倒今天依然适用。