1,021 questions
-1
votes
0
answers
9
views
Issue Uploading File to Stratus Bucket Using Node.js SDK
I am trying to upload my file to the Stratus bucket using the Node.js SDK method, but despite multiple attempts, the file is not uploading properly.
If someone can provide a working example of how to ...
- 1
0
votes
0
answers
7
views
Unable to Access Authenticated Bucket Object via Browser
I have created a Bucket in Catalyst with authentication enabled. After successfully inserting an object into the bucket, I tried accessing the object's URL in the browser, but I encountered the ...
- 1
0
votes
0
answers
19
views
Fetching an image and meta-deta from S3 and returning via API in a restful manner
I have an API that takes a time range input and fetches and returns the image back to the client via an API. One addition that makes this a bit tricky is that I also want to return some meta-data in ...
- 537
2
votes
1
answer
52
views
FastAPI and Google cloud Error 504 when uploading media
I've been facing this error on GCP for several days, where I get the 504 error on cloud however on the local machine it's working perfectly.
I'm using FastAPI for backend here is the endpoint :
@...
- 21
0
votes
0
answers
30
views
Accessing this api - GET https://storage.googleapis.com/storage/v1/b/bucket using api token only?
I wanted to access the api - GET https://storage.googleapis.com/storage/v1/b/bucket
with api token, is it possible, if yes, please let me know how to achieve that. Thanks in advance.
I used API KEY ...
- 11
0
votes
0
answers
16
views
How to get headings and sub headings for the user query along with the answer?
Im creating a generative chat bot in GCP with agend builder and Dialogflow cx.
The chat bot is giving good response. But I want the chat bot to get the headings and sub headings of the user query ...
- 1
1
vote
1
answer
53
views
I getting access denied error for bucket GCP
This is structure
From Load Balancer -> Bucket GCP
Bucket not public.
I need to deploy frontend on GCP bucket access only throgh Load balancer.
Like in AWS we only access S3 using Cloudfront not ...
- 11
0
votes
1
answer
64
views
Searching for a specific value, within a specific field, within many csv files, within many buckets, within 3 projects in GCS using Python
For example, let's say I want to search through my 3 projects > many buckets > many csv files > field that includes 'name' > for the value 'Bob'. So far I have the following code:
import ...
- 11
0
votes
1
answer
116
views
How to fix "AuthorizationHeaderMalformed when calling the Make Bucket operation" error in AWS s3 boto3
when i write this code it gives me this message, please help
aws s3 mb s3://hadsjkdshfjfdjhd
make_bucket failed: s3://hadsjkdshfjfdjhd An error occurred (AuthorizationHeaderMalformed) when calling the ...
0
votes
0
answers
46
views
How to, purposefully, delay the response from minio bucket
I am working on project where I am sending read requests on minIO bucket. I want to setup minio to delay its response to some specific time interval. How can I do so?
I want minio to delay its ...
- 1
0
votes
0
answers
88
views
Switch from s3 standard class to intelligent tiering
I want to completely move from standard class to intelligent tiering in s3 (change the class for older objects and the new class should be used for the objects created in future too). After some ...
- 5,094
0
votes
1
answer
351
views
Can't UPLOAD on s3 bucket with Laravel (always returns false)
I'm simply trying to upload anything on my AWS s3 bucket.
's3' => [
'driver' => 's3',
'key' => env('AWS_ACCESS_KEY_ID'),
'secret' => env('...
0
votes
1
answer
40
views
Google Signed Url Expiry issue
I am facing the issue with google signed URL expiry time. I want to signed URL expire in 5 seconds but its not expiring. here is my python code:
from datetime import datetime,timezone
...
-1
votes
1
answer
27
views
AWS selected Folders Transfer between buckets
Under My AWS Cloud S3 storage account i have around 55000 folders.
We are in process to reduce the size of the storage and to do that we have to move around 40000 folders.
What i need help from you ...
0
votes
0
answers
30
views
DAG: How can I Transform an unformatted file into a CSV in the same Bucket?
I'm trying to create a DAG (airflow) that processes an unformatted file uploaded to a GCP bucket and generates a new .csv file with specific transformations. The input file format is "CARTOLAS....
- 109
0
votes
0
answers
20
views
ANALYZE command not write data into hive metastore
Good day,
In SIT, I have manually create a hive table.
Follow by manually create a partition on it.
Then I copy the bucket file from other table and put inside the partition.
At this moment, if I run ...
- 1,903
-1
votes
1
answer
208
views
Gcs fuse with google buckets - Too many open files error with java app
I have a java spring boot application deployed as kubernetes application in GCP.
I use GCS fuse to access Google Cloud Storage buckets with the Cloud Storage FUSE CSI driver because it allows me to ...
- 21.4k
0
votes
0
answers
113
views
Why am I getting CORS error with 200 OK? (Cloud CDN with signed cookies)
This is Cloud CDN. I have created a bucket and a load balancer with a host and path rules like this:
Host: images.example.com Path: /* --> Bucket
I created an "A" DNS record image.example....
0
votes
0
answers
44
views
Mongodb $bucket with $min
I am trying to get stats on how many docs were created per period
[
{
$bucket: {
groupBy: "$createdAt",
boundaries: [
ISODate("2024-06-01"),
...
- 138
0
votes
0
answers
168
views
Can't upload file to a supabase bucket correctly with python
I am trying to upload a zipfile to a supabase bucket and update the root version, so the various devices can check for the latest "root" version and download the zip file if the version is ...
- 175
0
votes
1
answer
91
views
Google Cloud Storage Signed URL not Accessible in HTML but Works in Browser
`I'm using Python with the Google Cloud Storage library to generate signed URLs for my objects in a bucket. These URLs are meant to be embedded in HTML to display images on a webpage. However, while ...
0
votes
0
answers
37
views
Permission error when executing New-GcsObject in PowerShell
I'm encountering an issue when running a line of code in PowerShell from a user with only read permissions. The line in question is as follows:
New-GcsObject -Bucket "$Bucket" -...
0
votes
0
answers
28
views
Last user on a google cloud bucket, with limited rights
I'm "the last" activated user account of a company-own bucket that closed recently. As part of the closure, I'm in charge to back-up the data stored in this bucket. (important note: I'm ...
0
votes
1
answer
116
views
Is it safe for a Django backend's GCP storage bucket to be public?
I'm pretty confused by the permissions options for GCP's buckets. I have a Django Rest API that is dockerized and hosted on Cloud Run. I want the API to be accessible only by the client (which is ...
- 451
0
votes
0
answers
17
views
Binary not converting to image in browser(SPACES Digital Occean)
I recently moved to S3 to DigitalOccean Spaces and everything seems to work perfectly, but i noticed my images are not working correctly, every other type of archive is working just fine. My image ...
- 41
0
votes
1
answer
99
views
Create gc bucket with bash script
I want to automate the process of creating a Google Cloud bucket. For this, I have written a script that should do it. It should make a bucket to store media in from a dotnet appliation. Without the ...
-1
votes
1
answer
227
views
Spark Sort Operation in Join Despite Pre-sorted Bucketed Tables
I'm working with Spark and encountering an unexpected sort operation during a join of two pre-sorted and bucketed tables. Both tables have been created with the same number of buckets and are sorted ...
- 1
0
votes
0
answers
23
views
Import statements are "hiding" my js functions, What am I doing wrong? [duplicate]
The function uploadToBucket can no longer be referenced or found when adding the imports.
import { PutObjectCommand, S3Client, GetObjectCommand, ListObjectsV2Command } from ("@aws-sdk/client-s3&...
0
votes
1
answer
163
views
Bucket records into batches of a certain size in Snowflake
What would be the best way to bucket records into batches of a predefined size? I would like to tag each record with a batch/bucket number for further processing.
For example, let's say I have 1110 ...
- 5,295
2
votes
0
answers
230
views
move files between two folders in a GCS bucket
I have a DAG task like this:
move_files = GCSToGCSOperator(
task_id='move_files',
impersonation_chain='service_account',
source_bucket='ingestion_bucket',
source_object='ABCD/...
- 75
0
votes
1
answer
24
views
Making files inside any folder in my S3 bucket public
I am new with AWS and AWS S3 and I am not managing to see the files on certain folders in my bucket through their URLs.
My node.js application is uploading some static files to my folder bucket-name/./...
0
votes
2
answers
148
views
How to get the Data skipped from Storage Transfer Service Job (AWS -> GCP)
I was able to transfer data successfully using GCP Storage Transfer Service from my bucket in AWS to GCP, and I got that there are 17 files skipped because there are files that have the same name in ...
0
votes
1
answer
87
views
GCP entire Bucket exclusion - do we pay for each file deletion?
I know we have to pay for each file operation when we delete individual or mass objects in a bucket.
I have a bucket with around 15 million objects.
Does anyone know if deleting the entire bucket is ...
- 79
0
votes
0
answers
27
views
getting"api error AccessDenied: Access Denied" when making s3 bucket using terraform
I'm making s3 bucket with terraform to use it for static web site.
But I'm getting 'api error AccessDenied" error when I try to make s3 bucket.
The error message is below.
I searched for how to ...
- 1
0
votes
0
answers
61
views
how to create a ceph notification and receive object's changes?
I tried to get Ceph's message notification API, but found that it couldn't be turned on, so I ignored it, and then created a topic, created a rabbitmq message notification, but still couldn't receive ...
1
vote
0
answers
94
views
Behaviour differences between cburst and burst tc-htb linux
I've deployed an tc-htb in order to guarantee different bitrates to different flows. However, for traffic that belongs to class 1:10, I also want to allow some burst.
I've read the tc-htb ...
0
votes
1
answer
54
views
Minio S3 upload .jpg file to bucket
I need to create S3 bucket and upload .jpg files to created bucket.
public void saveTest(){
try {
MinioClient minioClient = MinioClient.builder()
.endpoint("http://...
- 15
0
votes
1
answer
255
views
KQL How to display default rows for every case even on no match
With KQL, is there a way to generate extra rows in the results table, for every possibility in a case statement, even if there were no matches from the dataset? This is to save the front end from ...
- 41
1
vote
1
answer
182
views
What is the better way of notifying my spring boot backend that an object has been stored into an AWS S3 bucket
I am currently trying to develop a "social media like" app as a personal project (to learn more)
I am currently designing the way my object storage will notify that an object has been ...
- 11
1
vote
1
answer
456
views
In Oracle Cloud Infrastructure, how to upload a csv file generated in a notebook directly into a bucket?
In my notebook, I generated a .csv file from a dataframe, and now I want to upload it into my bucket. How can I do that?
So far, I've only been able to generate the .csv, it appears in the file list ...
- 11
3
votes
1
answer
263
views
Compute similarity of two GPS track lines
I have ~2500 outdoor activities that I've recorded with a GPS tracker. I'd like to group similar activities automatically. Let's take a look at two related but not exactly similar activities:
One ...
- 8,727
0
votes
2
answers
479
views
HTB overlimits traffic BW
I am using TC Linux HTB to limit traffic bandwidths in the egress interface.
For it, I've set the following instructions on a linux container:
tc qdisc add dev eth2 root handle 1: htb default 12
tc ...
0
votes
1
answer
238
views
Load balance between GCP bucket and GKE
I have an GKE cluster with "Gateway" and "Httproute" resources like this.
kind: Gateway
apiVersion: gateway.networking.k8s.io/v1beta1
metadata:
namespace: development
name: dev-...
0
votes
1
answer
190
views
How can I configure Wordpress Media Cloud plugin using Oracle Cloud Bucket system
Altough i'm a frontend developer with experience, i'm barely a noob in what devops and cloud infrastructure means.
tldr;
I'm not sure how to fill the neccesary values to integrate a Media Cloud ...
- 44
0
votes
1
answer
63
views
Only want one file uploaded to bucket but two files are being uploaded to bucket every time I upload
Every time I upload a file to my bucket from my website two copies get uploaded. Is there anything wrong with my code that makes this happen?
const multer = require('multer');
const multers3 = require(...
0
votes
1
answer
131
views
Error - 11750 ## TwiML response body too large In your response to Twilio's request, the response body is larger than 64 kB
I'm trying to access an audio file which is stored in a GCP bucket from a TwiML template which has the following structure:
<?xml version="1.0" encoding="UTF-8"?>
<...
- 927
0
votes
1
answer
280
views
AWS S3 List Objects Access Denied On EC2
Iam new to AWS and now Iam facing the access denied problem when excuted
aws s3api list-objects --bucket BUCKET_NAME
Here are what I configured:
IAM role: attach AmazonS3FullAccess policy
EC2: ...
- 645
0
votes
1
answer
1k
views
supabase bucket getPublicUrl returns url that doesnt work
I'm using the supabase js API to create a public bucket and upload images to it.
This works so far:
const { data: bucket, error } = await supabase.storage.createBucket(
bucketName,
{
...
0
votes
0
answers
68
views
Sync Images from google server disk to Bucket
We are copying the images and documents from our local Ubuntu server to the Google bucket using the below command.
gsutil -m rsync -a public-read -R -C -f /media/data_disk/ITEM/ gs://dev-cdn-assets-...
- 61
0
votes
1
answer
139
views
Trying to send a message via linking an image to my slack channel from my s3 bucket
So i am using a lambda function to do this, but for some reason the image doesnt show, how can i display my images from s3? here is an example link: https://beconnectedtest-apex-home-...