[Python] LLM 을 이용하여 데이터 수집 및 요약 추출
LangChain과 OpenAI API 사용 / ChatOpenAI / StrOutputParser / ChatPromptTemplate / WebBaseLoader
Posted by
Wonyong Jang
on
July 18, 2024 ·
11 mins read
[Python] Python을 이용한 Crawling (Scrapy)
Crawling, Scraping / 사이트의 크롤링 정책
Posted by
Wonyong Jang
on
July 08, 2024 ·
5 mins read
[Python] Python을 이용한 Crawling (BeautifulSoup, Selenium)
웹 크롤링(Crawling), 웹 스크래핑(Scraping) / CSS Selector(태그 선택자, 클래스 선택자, ID 선택자)
Posted by
Wonyong Jang
on
June 24, 2024 ·
19 mins read
[Spark] Dynamic Partition Pruning / Speculative Execution
filter push down / dimension 테이블과 fact 테이블 조인시 쿼리 성능 최적화
Posted by
Wonyong Jang
on
May 15, 2024 ·
5 mins read
[Spark] Join Strategies 과 Shuffle
shuffle join, broadcast join / shuffle sort merge join, broadcast hash join
Posted by
Wonyong Jang
on
April 20, 2024 ·
7 mins read