Search DaniWeb - chunks

Re: The Application of Greedy Algorithm in Computer Monitoring Software 6 Days Ago by Dani > Maybe the tag system changed but in the past I could create a new tag if there wasn't one. Only non-newbie members can do that. Please visit our [tagging guidelines](https://www.daniweb.com/welcome/tagging). Retrieval Augmented Generation with Hugging Face Models in LangChain 2 Weeks Ago by usmanmalik57 … chunk_size=1000, chunk_overlap=200, length_function=len ) all_text_chunks = [] # To store chunks from all documents for doc in pages: text_content = doc.page_content… any vector store index such as `FAISS` to store embedded chunks. ``` model_path = "thenlper/gte-large" embeddings = HuggingFaceEmbeddings(… Question Answering with YouTube Videos Using RAG in LangChain 3 Weeks Ago by usmanmalik57 … and prepare them for retrieval, we split them into smaller chunks using the LangChain `RecursiveCharacterTextSplitter`. This facilitates more efficient processing and…. The embeddings are generated using `OpenAIEmbeddings,` which transform the text chunks into numerical vectors. These vectors are stored in a FAISS… Retrieval Augmented Generation (RAG) with Google Gemma From HuggingFace 2 Months Ago by usmanmalik57 …45 ``` Next, we will divide our documents into multiple chunks using the `RecursiveCharacterTextSplitter` from the `langchain.text_splitter` module… 1000 characters with an overlap of 200 characters between all chunks. ``` splitter = RecursiveCharacterTextSplitter( chunk_size=1000, chunk_overlap=200, length_function=… Paris Olympics Ticket Information Chatbot with Memory Using LangChain 1 Month Ago by usmanmalik57 … split our PDF document and create embeddings for the different chunks of information in the PDF document. We store the embeddings… Re: libpng: custom chunks 13 Years Ago by Jamesbch … chunk handling stuff until after png_read_update_info(), so all the chunks before IDAT have already been read. Try moving line…as if it were unknown. In pngtest.c, the unknown chunks (vpAg and sTER) really are unknown to libpng.[/QUOTE]…png_set_read_user_chunk_fn[/I] so I though it wasn't handling custom chunks. I've changed the [I]return[/I] value … libpng: custom chunks 13 Years Ago by Jamesbch …i x %i\n", width, height); /* ignore all unknown chunks: */ png_set_keep_unknown_chunks(png_ptr, 1, NULL, 0); /* except for our …png_ptr, 2, custom_chunk, sizeof(custom_chunk)/5); /* also ignore unused known chunks: */ png_set_keep_unknown_chunks(png_ptr, 1, unused_chunks, sizeof(unused_chunks)/5); png_set_read_user_chunk_fn(png_ptr… Re: libpng: custom chunks 13 Years Ago by Glenn R-P … your chunk handling stuff until after png_read_update_info(), so all the chunks before IDAT have already been read. Try moving line 85…) as if it were unknown. In pngtest.c, the unknown chunks (vpAg and sTER) really are unknown to libpng. Reading a binary file in chunks... 14 Years Ago by bobjonson4 … response object found in ASP.DLL, sends the file in chunks of 96kb. The aim was for me to create code… required"? Like not all at once, only read the chunks when needed and send. Hope that makes sense! [CODE] Dim… Re: libpng: custom chunks 13 Years Ago by Glenn R-P Your problem may be due to declaring IDAT an unknown chunk. Libpng needs to see at least one IDAT chunk before it can do png_read_end() which reads any chunks after IDAT, including the IEND chunk. Glenn Re: libpng: custom chunks 13 Years Ago by Jamesbch The solution is to not ignore the IDAT and critial chunks of the file and to read the IDAT even if … Write an output file by fixed length chunks. 12 Years Ago by Gribouillis When working whith large data files, it may be desirable to output a sequence of bytes by large chunks. This snippet defines a file adapter class to handle this transparently. Writing bytes to an ChunkedOutputFile will automatically write the underlying file object by fixed length chunks. Re: Write an output file by fixed length chunks. 12 Years Ago by Gribouillis …;""Adapter class to write an output file by chunks of fixed length. Typical use: # This code writes a sequence… of strings to myfile.txt by chunks of 32 megabytes. with open("myfile.txt", "… Read by chunks from binary with java 9 Years Ago by Garidius … some java expert could help me. How can I read chunks from binary file each chunk at a time? The… chunks are separated with the bytes D2 followed by A7 and … Re: Read by chunks from binary with java 9 Years Ago by Garidius … and result 4 sequences of D2A7, resulting in 3 complete chunks and 1 incomplete chunk (224 bytes). 2- Then, from those… How to get data from GetListItems of lists.asmx in chunks?(For eg: 100 at a time) 12 Years Ago by vivek4020 Hi, I am trying to get data from GetListItems in chunks of 100 so that processing them can be easier. i can specify the row limit like this. <rowLimit>100</rowLimit> But how do i get the next 100 items? Is there any way to skip over the first 100 items? Vivek Re: libpng: custom chunks 13 Years Ago by Jamesbch Unfortunately it isn't as easy as it seems to be. I've changed (diff) : [code=diff]18c18 < 'I', 'D', 'A', 'T', (png_byte) '\0' /* IDAT */ --- > //'I', 'D', 'A', 'T', (png_byte) '\0' /* IDAT */ 102c102 < //png_read_end(png_ptr, end_info); --- > png_read_end(png_ptr, end_info); [/code] But : [quote]$ ./png-embedder -r file2.png … Re: libpng: custom chunks 13 Years Ago by Glenn R-P This discussion in continued on the png-mng-implement at sourceforge.net list Glenn Re: libpng: custom chunks 13 Years Ago by Jamesbch Yes thank you, I didn't know you were part of the official ML. I will post the result here when I've finished reading all the mails. Re: Write an output file by fixed length chunks. 12 Years Ago by TrustyTony This snippet could probably be good foundation for file copy/installation with progress bar by adding function to call for each block. It could take parameters for current block number and total number of blocks. Re: Write an output file by fixed length chunks. 11 Years Ago by TrustyTony What is the benefit of using this snippet compared to io module's buffered io? Re: Write an output file by fixed length chunks. 11 Years Ago by Gribouillis > What is the benefit of using this snippet compared to io module's buffered io? You are right, there may be no advantage at all. I didn't think of that when I wrote the snippet. Re: Read by chunks from binary with java 9 Years Ago by JamesCherrill Is the rest of the file just text? If so, just read the file using a Scanner with your D2A7 bytes as the delimiter https://docs.oracle.com/javase/7/docs/api/java/util/Scanner.html Re: Read by chunks from binary with java 9 Years Ago by Garidius Hello James, Actually all the content of the file is binary, but if the scanner is able to use delimiter such as the sequence 0xD20xA7 and return each token as documentation says, I'd be able to convert that token into hexadecimal string. I'm trying with code below, but I cannot test the code since appear a message that says "No suitable … Re: Read by chunks from binary with java 9 Years Ago by JamesCherrill There's nothing wrong with new Scanner(File), but line 10 has a new Scanner(Scanner) and that's completely wrong (and unnecessary). Also, the the delimiter is wrong. Your delimiter is not the String "0xDA0xA7", it should be a String of length 2. You need something like byte[] delimBytes = {(byte) 0xda, (byte) 0xa7}; String … How can i hold the graph data upto 10 minutes in vb.net 14 Years Ago by san.kothari …' Paint Guidelines on picGraph picGraph.Image = DisplayGuidelines(picGraph, Chunks) ' Paint Guidelines and Numbers on picValues picValues.Image =… strValue = port2.ReadExisting 'picGraph.Image = DisplayGuidelinesAndChart(picGraph, Chunks, XMove, strvalue, CInt(txtMin.Text), CInt(txtMax.Text))… Struct syntax problem? 15 Years Ago by death_oclock … int nProps; GROUP *forms; int nForms; CHUNK *chunks; int nChunks; }; void groupCon(GROUP *group); void…props); free(group->forms); free(group->chunks); } void chunkCon(CHUNK *chunk) { //chunk-&… match any IDs, is a chunk { realloc(current->chunks, ++current->nChunks * sizeof(CHUNK)); } } } … Extracting Information from Research Papers Using Langchain & OpenAI 4 Months Ago by usmanmalik57 …` from the Langchain module to divide the text into manageable chunks. ``` splitter = CharacterTextSplitter( separator = "\n", chunk_size =… len, ) text_chunks = splitter.split_text(pdf_text) print(f"Total chunks {len(text_chunks)}") print("============================") print(text_chunks[0]) ```… Python FFT finding frequencies-Numpy 13 Years Ago by Masood_786 … p = pyaudio.PyAudio() chunks=[] ffts=[] def stream(): global chunks, inStream, bufferSize while True: chunks.append(inStream.read(bufferSize)) … def graph(): global chunks, bufferSize, fftx,ffty, w if len(chunks)>0: data = chunks.pop(0) data=scipy… Re: Python FFT finding frequencies-Numpy 13 Years Ago by Masood_786 …=48100 #sampleRate=64000 p = pyaudio.PyAudio() chunks=[] ffts=[] def stream(): global chunks, inStream, bufferSize while True: chunks.append(inStream.read(bufferSize)) def record… [lx,ly] def graph(): global chunks, bufferSize, fftx,ffty, w if len(chunks)>0: data = chunks.pop(0) data=scipy.array(struct…