How to Use Local ChatGPT GPT4All

In the last blog, we learned how to install and run Ollama-powered privateGPT. In this guide, we will further learn how to use local ChatGPT GPT4All, introduce its interface features, and try to chat with local documents.

There are many reasons to use GPT4All, an alternative to ChatGPT, including:

You can run GPT4All using your PC's CPU, and you don't need a powerful but expensive GPU and more than 12 GB of VRAM (although it can help).

GPT4All is a completely offline solution, so it can be used even if you don't have access to the internet.

Being offline and running as a "local app" also means that all the data you share with it stays on your computer, and its creator can't "peep into your chats."

Since it relies solely on your PC, it won't slow down, stop responding, or ignore your prompts like ChatGPT does when its servers are overloaded.

GPT4All allows you to easily add documents to its "knowledge base". It can then render information from them in answers, turning it into a true personal assistant.

After installing GPT4All and downloading some LLMs, you can now start chatting with your own AI chatbot.

Use the drop-down menu at the top of the GPT4All window to select the active language model.

Use the hamburger icon in the upper left corner to access GPT4All's control panel.

The first option on the GPT4All panel allows you to create a new chat, rename the current chat, or delete it.

The Update button allows you to update the application and add or remove components.

The Download button gives you access to the Language Model panel.

You can use the Language Models panel to install new LLMs, remove installed LLMs, and change their download paths.

Chatting with LLM in GPT4All is similar to the online version of ChatGPT. Enter something into the input field at the bottom of the GPT4All window and press Enter, and you'll see a prompt in GPT4All's main view. The selected language model's response will appear below the prompt.

As with ChatGPT, if you don't like the LLM response generated on the screen, you can click the Stop Generation button to interrupt the process.

Likewise, if you don't like the generated response and want to see alternatives, you can click the "Regenerate Response" button that appears in the same location.

Note that GPT4All also gives you access to so-called unrestricted language models. There are no artificial restrictions on these. This “freedom” is said to “liberate” and “empower” them to be more creative and provide more factual responses. However, this also means they are likely to produce offensive output.

The first two buttons in the upper right corner of the app allow you to reset the conversation or copy it to the clipboard to paste it elsewhere.

The third button (with the gear icon) gives you access to GPT4All’s settings.

The Models tab of the GPT4All settings allows you to configure the parameters of the active language model. By changing variables like temperature and repetition penalty, you can tweak its creativity, simplicity, and more.

The Application tab allows you to select the default model for GPT4All, define the download path for language models, allocate a specific number of CPU threads to the application, automatically save each chat locally, and enable its internal web server to make it Accessible via browser.

The best feature of GPT4All is that it makes it easy to add your own documents to the language model of your choice. As we've seen, you can do the same thing with ChatGPT and build a custom ChatGPT with your own data. However, with GPT4All, the process is much easier and comes without the cost of using Open AI’s ChatGPT API.

Put some documents into a folder. Make sure they are in widely compatible file formats such as TXT, MD (for Markdown), Doc, etc.

On the GPT4All Settings panel, go to the LocalDocs tab page. Click the Browse button and point the application to the folder where you placed the document. Then click Add to include them in GPT4All's list of external documents.

You will see an entry for the document folder in the document list of GPT4All's LocalDocs plugin. If you decide to delete a Documents folder, select it from here and click the Delete button to its right.

From the same location, you can adjust the document fragment size and document fragment to handle each prompt value in the lower left corner of the panel. Increasing these will improve the chosen language model's ability to "understand" your document and provide more relevant responses. However, it also severely slows down GPT4All, since it has to analyze your document after each prompt.

You can manage your external documents through GPT4All's Collections button.

You can add multiple document collections to GPT4All. However, it is better not to leave them active all the time, but to control which document groups ("collections") are activated from the Collections panel by reducing the performance of GPT4All.

After you add some documents and enable their collection, GPT4All will tell you after a prompt that it is processing them.

Every time GPT4All provides a response that extracts data from a document, it provides a reference list and contextual links under the response.

By clicking on the contextual link, GPT4All will show you the text snippet from the document for the response.

GPT4All and the language models you can use with it may not exactly match the dominant ChatGPT, but they are still useful. Creative users and tinkerers have found all kinds of ingenious ways to improve such models so that even if they rely on smaller datasets or slower hardware than what ChatGPT uses, they can still get close to it, or in In some cases even beyond it. But rather than read in detail about the pros and cons of each available model, why not try it out for yourself? GPT4All makes it easy.