John Davidson

Message:

I would like to validate a large (up to millions of rows) CSV file content without reading it twice : once in the validation request file, once when storing the content of the CSV file to the database. What's the best, clean and efficient approach?

Here is what I have so far:

CreateSymbolRequest.php:

public function rules()

{

    return [

        'prices' => [

            'bail',

            'required',

            'file',

            'max:8192',

            new PriceFileExtensionsRule(['csv']),

            new PriceFileHeadersRule,

            new PriceFileDataFormatRule

        ],

        'pair' => 'required|array',

        'timeframe' => 'required|in:1S,1,1H,4H,1D,1W,1M',

        'type' => 'required|in:spot,future',

        'exchange' => 'required|in:FTX,binance',

        'tx_fee' => 'required|numeric',

        'm_fee' => 'required|numeric'

    ];

}



public function persist(): void

{

    // checks if the Symbol already exists in the DB

    if ($this->isSymbolDuplicate()) {

        throw new Exception('Symbol already exists in the database.');

    }



    // Stores the Symbol in the DB

    $this->persistSymbol();

    // Stores the CSV content in the DB

    $this->persistPrices();

}

SymbolsController:

public function store(CreateSymbolRequest $request)

{

    $request->persist();    

}

Of course, I could split the basic validation in CreateSymbolRequest.php and the more advanced validation in SymbolsController.php, but that would kind of defeat the purpose of having a separate Request file with the validation rules in it.

I was also thinking of passing a $content variable by reference to the Rule files but I don't think I can return anything from there and that feels very ugly anyway.

---

My Answer:
'required|file|mimes:csv|max:2048', ];}_CreateSymbolController.php:_public function store(Request $request)_{_ $validatedData = $request->validate([ 'prices' => 'required|file|mimes:csv|max:2048', ]); $file = $request->file('prices'); $csv = array_map('str_getcsv', file($file)); // Store $csv data to database_}